1. How to Create a Rubric Presented by the ORIE Team How to Create a Rubric

2. Participants’ Learning Outcomes At the end of this training section, participants will be able to do the following:  Determine which types of assignments need to use a rubric for grading;  Design a rubric for grading student work.

3. What is a rubric? A rubric is a systematic scoring guide to evaluate student performance. It specifies levels of quality in identified areas.

4. What types of assignments require the use of a rubric? A rubric is needed if an assignment entails varied responses, rather than one correct answer:  Essay  Essay questions on an exam  Research paper  Oral presentation  Portfolio  Demonstration of critical thinking skills  Demonstration of technical skills  An Art Show, etc.

5. What doesn’t require the use of rubrics? If assignments or exam questions entail only right or wrong answers, then there is no need for a rubric. A score distribution guide indicating the distribution of points for correct answers is used instead.  Math questions  Multiple-choice questions  True or false questions  Filling-in-the-blanks test  Matching exercise or test

6. Discussion Which of the following assignments requires the use of a rubric? (There are 3 of them.)  A welding project, using appropriate design and materials to weld the common joints  Multiple-choice questions on an exam  Students’ self-report on their experience with service learning  Clinical performance  True or false questions demonstrating students’ knowledge of medical law  Final exam with a combination of multiple-choice questions and matching lists.

7. How to create a rubric? There are many kinds of rubrics, but the most commonly used rubric is the descriptive rubric. A descriptive rubric should include at least three essential parts: 1)A Set of Indicators of Learning 2)A Scale 3)A Set of Descriptors

8. Identifying Indicators of Learning Indicators of Learning are traits or features of an assignment that an instructor wants to make judgment on. Indicators of Learning should be aligned with the instructional objectives. If an instructor’s objectives of teaching an essay are related to helping students compose an essay that is focused, coherent, organized, etc., then these features should serve as indictors of learning. Example 1: Indicators of Learning for an Essay for an English class:  Focus  Coherence  Organization  Sentence Structure  Word Choice

9. Identifying Indicators of Learning Example 2: Indicators of Learning for Critical Thinking Skills:  Explanation of issues  Evidence  Influence of context and assumptions  Student’s position (perspective, thesis/hypothesis)  Conclusions and related outcomes (implications and consequences) Example 3: Indicators of Learning for an Oral Presentation:  Organization  Subject Knowledge  Graphics  Mechanics  Eye Contact  Elocution

10. Creating a Scale A scale indicates points to be assigned in scoring a piece of work on a continuum of quality. High numbers are typically assigned to the best work. Scale examples: Needs Improvement (1)... Satisfactory (2)... Good (3)... Exemplary (4) Beginning (1)... Developing (2)... Accomplished (3)... Exemplary (4) Needs work (1)... Good (2)... Excellent (3) Novice (1)... Apprentice (2)... Proficient (3)... Distinguished (4)

11. Creating a Descriptor Descriptors are used to describe signs of performance at each level; the description needs to include both strengths and weaknesses (Weaknesses should be described particularly in lower levels of performance). Example 1: Word Choice (in an essay): 4-- Vocabulary reflects a thorough grasp of the language appropriate to the audience. Word choice is precise, creating a vivid image. Metaphors and other such devices may be used to create nuanced meaning. 3-- Vocabulary reflects a strong grasp of the language appropriate to the audience. Word choice is accurate, but may be inappropriate in a couple of places. 2-- Vocabulary reflects an inconsistent grasp of the language and may be inaccurate or inappropriate to the audience. 1-- Vocabulary is typically inaccurate and inappropriate to the audience. Word choice may include vague, non-descriptive, and/or trite expressions.

12. Creating a Descriptor Example 2: Depth of Discussion (in a research paper) 4– In-depth discussion and elaboration in all sections of the paper 3– In-depth discussion and elaboration in most sections of the paper 2-- The writer has omitted pertinent content, or content runs on excessively. Quotations from others outweigh the writer’s own ideas excessively. 1– Cursory discussion in all the sections of the paper or brief discussion in only a few sections.

13. Example of a Rubric CapstoneMilestonesBenchmark Indicators4 3 21 Explanation of Issues Issue/problem to be considered critically is stated clearly and described comprehensively, delivering all relevant information necessary for full understanding Issue/problem to be considered critically is stated, described, and clarified so that understanding is not seriously impeded by omissions. Issue/problem to be considered critically is stated but description leaves some terms undefined, ambiguities unexplored, boundaries undetermined, and/or backgrounds unknown. Issue/problem to be considered critically is stated without clarification or description. Evidence Information is taken from source(s) with enough Interpretation/ evaluation to develop a comprehensive analysis or synthesis. Information is taken from source(s) with enough interpretation /evaluation to develop a coherent analysis or synthesis. Information is taken from source(s) with some interpretation/ evaluation, but not enough to develop a coherent analysis or synthesis. Information is taken from source(s) without any interpreta- tion/evaluation. AACU Critical Thinking VALUE Rubric (A Snapshot)

14. Example of a Rubric A Research Paper Rubric (A Snapshot) Indicator EXPERTPROFICIENTAPPRENTICENOVICE TOPIC FOCUS The topic is focused narrowly enough for the scope of this assignment. A thesis statement provides direction for the paper, either by statement of a position or hypothesis. The topic is focused but lacks direction. The paper is about a specific topic but the writer has not established a position. The topic is too broad for the scope of this assignment. The topic is not clearly defined. DEPTH OF DISCUSSION In-depth discussion & elaboration in all sections of the paper. In-depth discussion & elaboration in most sections of the paper. The writer has omitted pertinent content or content runs on excessively. Quotations from others outweigh the writer’s own ideas excessively. Cursory discussion in all the sections of the paper or brief discussion in only a few sections.

15. Example of a Rubric An Oral Presentation Rubric (A Snapshot) Indicators 1234 Eye Contact Student reads all of report with no eye contact. Student occasionally uses eye contact, but still reads most of report. Student maintains eye contact most of the time but frequently returns to notes. Student maintains eye contact with audience, seldom returning to notes. Elocution Student mumbles, incorrectly pronounces terms, and speaks too quietly for students in the back of class to hear. Student's voice is low. Student incorrectly pronounces terms. Audience members have difficulty hearing presentation. Student's voice is clear. Student pronounces most words correctly. Most audience members can hear presentation. Student uses a clear voice and correct, precise pronunciation of terms so that all audience members can hear presentation.

16. Hands-on Activity Create a Rubric for an Assignment, using the following chart: Indicators 1234

17. Resources AACU Publication Webpage: http://www.aacu.org/value/publications.cfm http://www.aacu.org/value/publications.cfm Suskie, L. (2009). Assessing Student Learning. San Francisco: Jossey-Bass. UT-Austin Assessment Webpage: https://www.utexas.edu/provost/planning/assessme nt/iapa/workshops.html