Rubrics for Complex Papers/Projects Academic Assessment Workshop May 14-15, 2009 Bea Babbitt, Ph.D.

Complex Project  Can be used to gather information on multiple learning outcomes for the course or program.  Examples include end-of-semester paper or project, semester-long project, research study, clinical experience.

Assessment Definition  Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance.  ~Tom Angelo, AAHE Assessment Forum

Can you answer the following questions?  What do you want students to learn?  How well do you want them to learn it?  What evidence of learning will you require?  Do students understand your expectations?  How well do individual students or groups of students perform?  What’s the relationship between instruction and various components and levels of performance?  When performance doesn’t match expectations, what actions can be taken to improve performance?  Do the actions taken result in improved performance?

10 sure signs you need a rubric  You are getting carpal tunnel syndrome from writing the same comments on almost every student paper.  You have graded all your papers and worry that the last ones were graded slightly differently from the first ones.

10 signs  You want students to complete a complex assignment that integrates all the work of the term and are not sure how to communicate all the varied expectations easily and clearly.  You give a carefully planned assignment that you never used before, and to your surprise, it takes the whole class period to explain it to students.

10 signs  Students ask many questions about the long narrative description of the assignment in the syllabus.  Students can’t explain the assignments or expectations to the Writing Center or other tutorial services.  All or most of your class were unaware of academic expectations so basic you neglected to mention them (e.g., spell check, citations)

10 signs  Students provide many excuses why they couldn’t possibly meet your expectations; you’d like to say, “here are the standards, you’re an adult, you weigh all your responsibilities and choose how well you would like to do”.  You’ve designed a common assignment across course sections, but aren’t sure faculty are using a common grading scale.

What’s a rubric?  Scoring tool  Lays out specific expectations for an assignment  Divides an assignment into its component parts  Provides a detailed description of what constitutes acceptable or unacceptable levels of performance for each of those parts

At what level will you assess?  Level 1. Assessing individual student learning within courses  Level 2. Assessing individual student learning across courses  Level 3. Assessing courses  Level 4. Assessing programs  UNLV Program Assessment: Includes aspects of both 3 & 4  Level 5. Assessing the institution

Why rubrics for complex papers/projects?  Defines major components of paper/project  Communicates expectations  Conveys feedback to students  Saves grading time  Insures cross-section/cross-instructor scoring consistency (with training, of course)  Facilitates aggregating scores across students/sections to determine group performance on each skill component, or program or general education objective.

Parts of a Rubric  Task description (the assignment)  Scale (e.g. Exceeds expectations, meets expectations, below expectations)  Dimensions (a breakdown of the skills/knowledge demonstrated through the assignment)  Descriptions (describes the dimension in detail at each level of the scale) set out on a grid (matrix).

Basic Rubric Grid Format Scale Level 1 Scale Level 2 Scale Level 3 Scale Level 4 Dimension 1 Dimension 2 Dimension 3 Dimension 4 Dimension 5 Assignment Title Task Description:

Rubric with Scale: High to Low ExemplaryProficientBasicNovice Dimension 1 Dimension 2 Dimension 3 Dimension 4 Dimension 5 Short Essay Task Description (complete):

Add Dimensions ExemplaryProficientBasicNovice Structure Analysis Evidence Add dimension Short Essay Task Description (complete):

Out-to-lunchGlimmer of understandin g So-soName this level StructureOne paragraphThree or more paragraphs, arbitrarily divided, word music rambling in a thick haze. Multiple paragraphs adhering to some formulaic notion of an essay -- intro., body, conclusion. Originality. Structure driven by discourse. AnalysisThe lights are completely out. It's a power failure There is mild evidence of thought. The obvious and banal are represented as great insight Depth of understanding beyond the obvious but is weighted down with questionable logic, gross assumptions, and a lack of perspective. Original thought, challenging assumptions, and shedding new light on old subjects. EvidenceWhat evidence? It's all unsupported claims. Perfunctory, sporadic support is juxtaposed with unsubstantiated assumptions and bromides. The occasional citation is incomplete Most claims are supported with evidence, properly cited; reliable sources are given weight equal to internet crackpot claims. The analysis is driven by the evidence which is extensive and from a variety of reliable sources, all of them properly and thoroughly cited. Etc. Short Essay Task Description: Partial Rubric (Humorous) Write Descriptions

Examples, examples, examples

Starter Resources  Internet Resources for Higher Education Outcomes Assessment  Miller, R., & Leskes, A. (2005). Levels of assessment: From the student to the institution. Washington, D.C.: Association of American Colleges and Universities.  UNLV Assessment Home Page  Winona State University Sample Rubrics