1. Using RUBRICS to Assess Program Learning Outcomes By Dr. Ibrahim Al-Jabri Director, Program Assessment Center April 16, 2007

2. OUTLINE Questions?? Assessment Cycle Direct & Indirect Measures What is rubric? Type of rubrics Constructing Rubrics Using Rubrics Advantages of Rubric Examples

3. Questions?? What do we want students learn? Do students have the opportunity to learn it? How do we know that they learned it? What do we do with that information?

4. Assessment Cycle W rite expected outcomes/objectives E ffect improvements through actions V iew results A ssess performance against criteria E nsure students have opportunity to learn

5. Direct and Indirect Measures of Student Learning Direct measures: –“… tangible, visible, self-explanatory evidence of exactly what students have and haven’t learned” (Suskie, 2004, p. 95). Indirect measures: –“… signs of what students are learning, but the evidence of what they are learning is less clear and less convincing” (Suskie, 2004, p. 95).

6. Direct and Indirect Measures of Student Learning Direct –Tests –Rubrics –Portfolios –Capstone projects –Field supervisor ratings –Employer ratings Indirect –Course grades –Surveys –Focus Groups –Student self ratings –Course evaluations –Graduate school admissions

7. One Useful Direct Measure of Student Learning in Courses or Programs is Rubric

8. What is Rubric? Rubrics offer the Instructor an opportunity to assess the student's understanding of a scientific topic by levels of performance on certain criteria

9. Type of rubrics Rubrics may be used “holistically” or “analytically”… “Holistic” Rubric: –The entire response is evaluated and scored as a single performance category “Analytical” Rubric: –The response is evaluated with multiple descriptive criteria for multiple performance categories

10. Template for analytic rubrics Beginning 1 Developing 2 Accomplished 3 Exemplary 4 Score Criteria #1 Description reflecting beginning level of performance Description reflecting movement toward mastery level of performance Description reflecting achievement of mastery level of performance Description reflecting highest level of performance Criteria #2 Description reflecting beginning level of performance Description reflecting movement toward mastery level of performance Description reflecting achievement of mastery level of performance Description reflecting highest level of performance Criteria #3 Description reflecting beginning level of performance Description reflecting movement toward mastery level of performance Description reflecting achievement of mastery level of performance Description reflecting highest level of performance

11. Proficient- 3 points The student's project has a hypothesis, a procedure, collected data, and analyzed results. The project is thorough and the findings are in agreement with the data collected. There are minor inaccuracies that do not affect the quality of the project. Adequate- 2 points The student's project may have a hypothesis, a procedure, collected data, and analyzed results. The project is not as thorough as it could be; there are a few overlooked areas. The project has a few inaccuracies that affect the quality of the project. Limited - 1 point The student's project may have a hypothesis, a procedure, collected data, and analyzed results. The project has several inaccuracies that affect the quality of the project. Example for holistic rubrics

12. Constructing a Rubric –Know the goals for instruction- what are the learning outcomes? –Decide on the structure of the rubric- holistic or analytical--what fits best for the task? –Determine the levels of performance- are there levels of performance specific to each criteria? –Share the rubric with your students-students should have an opportunity to see, discuss or even design the rubric.

13. What are the steps in creating a rubric? 1.Define the focus / topic / task / learning objective 2.List the criteria for assessing the performance 3 - 8 characteristics Clearly written with action verbs 3.Write descriptions for the levels of performance Usually 3 - 5 levels Define top level first, then bottom level, and finally middle level(s) 4.Test rubric

14. Using Rubrics Useful in other ways: Self-assessment Peer evaluations Analysis of electronic discussion threads Employer ratings of recent graduates Program assessment, giving direct evidence of student learning – most common use Embedded course assignments Capstone experiences Field experiences Portfolios

15. Using rubrics in assessment Formative assessment: Rubrics can be used as ongoing ways of measuring student progress. This feedback provides direction for further teaching. Summative assessment: Rubrics can be used to measure achievement of specific objectives, competencies, or learning goals.

16. Advantages of Rubrics Help students know what to expect Provide detailed feedback Help students see improvement over time Offer guidance for continuous improvement Align teaching and learning to standards

17. Examples

19. Oral Presentation Rubric Criteria0123 Eye Contact Does not attempt to look at audience at all, reads notes the entire time Only focuses attention to one particular part of the class, does not scan audience Occasionally looks at someone or some groups during presentation Constantly looks at someone or some groups at all times Facial Expressions Has either a deadpan expression of shows a conflicting expression during entire presentation Occasionally displays both a deadpan and conflicting expression during presentation Occasionally demonstrates either a deadpan OR conflicting expression during presentation Gives audience clues to what the content of speech is about; Appropriate expression, never notice a deadpan or conflicting expression Enthusiasm Shows absolutely no interest in topic presented Shows some negativity toward topic presented Occasionally shows positive feelings about topic Demonstrates a strong positive feeling about topic during entire presentation Visual Aid Poor, distracts audience and is hard to read Adds nothing to presentation Thoughts articulated clearly, but not engaging Visual aid enhances presentation, all thoughts articulated and keeps interest Completeness of Content One or more points left out Majority of points glossed over Majority of points covered in depth, some points glossed over Thoroughly explains all points Professionalism of Presentation Audience has difficulty hearing, confusing Thoughts don't flow, not clear, does not engage audience Thoughts articulated clearly, though does not engage audience Presentation is organized and the interest level of the audience is maintained

20. Thank You