Using Rubrics for Assessment: A Primer Marcel S. Kerr Summer 2007 

Presentation Purpose   Educate faculty on uses of rubrics  Preparation for General Education Assessment Plan Learn to write rubrics for multiple purposes  Student learning assessment  Course assessment  Program assessment

Presentation Objectives   Define rubric  Differentiate types of rubrics  Identify uses of rubrics  List advantages of using rubrics  Construct a rubric  Determine validity & reliability  End-of-training Quiz  Complete Rubric Training Rubric

What are Rubrics?   A rubric is a scoring scale used to assess performance along a task-specific set of criteria rather than a single numeric score.  A means of formative assessment  Purposes of rubrics: Describe what “outstanding” or “satisfactory” or “inadequate” performance looks like Articulate the criteria against which student learning is judged Provide an objective guide for assessment

What do Rubrics Assess?  Assessment level: Student learning Courses Programs Institutions  A stated learning objective: Content – the knowledge specific to a given discipline Performance – skills and internal processes required for achievement in a given discipline

Types of Rubrics   Analytic – are more specific & easier to apply; break down the objective into dimensions (criteria) and judge them individually Used when large number of criteria; criteria are weighted  Holistic – are more general; assess the entire objective on one scale Used for quick or minor assessments; open-ended questions; skills

Rubric Basic Structure   There are four basic features:  Stated Objective or Purpose - title  Scoring Criteria – characteristics of good performance on the task; Levels of Performance – defined degrees of competency  Descriptors – spell out briefly what is expected at each level of performance

Rubric Basic Structure  Criteria123 Number of Sources Historical Accuracy Lots of historical inaccuracies Few inaccuraciesNo apparent inaccuracies Organization Cannot tell from which source information came Can tell with some difficulty where information came Can easily tell from which sources information was drawn Use of APA Format Lots of APA errorsFew APA errorsNo apparent APA errors Objective: Research Paper

Uses of Rubrics   Set evaluator & performer expectations  Criteria by which work is judged  Difference between excellent & weak work  Formative student feedback  Grade assignments  Standardize grading across graders  Assess programs (GEC)

Rubrics for Program Assessment   Rationale for use: Faculty created, owned, and operated Low cost & reusable National tests are expensive and problematic Natural part of the course or curriculum  Four General Education Literacy areas: Language Literacy – Written CommunicationWritten Communication Analytic Literacy – Problem SolvingProblem Solving Cultural Literacy – Composition of ArtComposition of Art Social Literacy – Personal & Civic ResponsibilityPersonal & Civic Responsibility

Advantages of Using Rubrics   Clarity Expectations, objectives, grading, feedback  Objectivity Standardized, consistent, fair, valid, reliable  Legitimacy Fairness increases student responsibility  Efficiency Easy to make, use and explain  Improve skills & End Products Instructor, students, peers

Writing Rubrics   Identify and define the assessment objective or purpose  Select and write the needed number of scoring criteria Select and write the desired levels of performance  If desired, select and write the descriptors

Electronic Rubric Builders  Here are a few: Teachnology.com Rubistar Rubric Studio

Calibrating Rubrics: Validity   Validity – the accuracy with which the rubric assesses the objective or purpose; are we evaluating what we intended? Self Check Colleague Review Student Review Pilot Test

Calibrating Rubrics: Reliability   Reliability – how consistently the rubric assesses the objective or purpose over time and across raters; are the resulting scores consistent? Interrater – consistency of scores assigned by two or more raters Intrarater – consistency of scores assigned by same rater at different points in time  External factors – time of day, order of assignments  Internal factors – mood, fatigue, attention

End-of-training Quiz  What have we learned? Please complete the 10-item online quiz. It is a quick formative assessment of what we’ve covered today. Upon completion, you will receive a feedback page with your final score. Please print a copy of this page and forward it to your Dean or VP as evidence that you have completed the assessment training.

Rubric Resources  Documenting Excellence – General Education Rubrics: /rubric.htm /rubric.htm  Authentic Assessment Toolbox:  Rubrics.com:  Teacher Created Rubrics for Assessment:  Sinclair Community College – General Education Rubrics:  CSU – Institutional research Assessment & Planning:

References  Moskal, B. M., & Leydens, J.A. (2000). Scoring rubric development: Validity and reliability. Practical Assessment, Research & Evaluation, 7(10). Retrieved June 19, 2007 from  Practical Assessment, Research and Evaluation. A Peer-reviewed electronic journal. ISBN  Richards, K. (2005). A primer on rubrics. Retrieved, June 15 th, 2007 from  Stevens, D.D., & Levi, A.J. (2004). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback and promote student learning. Stylus Publishing.