1. Using Rubrics to Evaluate Assessments Kim Anderson Course Evaluation Subcommittee Chair Summer 2009

2. What is a Rubric? A rubric is a continuum of scoring categories and guidelines for teachers to evaluate and to give feedback to individual students AND to evaluate groups of student work to assess for course effectiveness.

3. Uses of a Rubric Within a class –Clarity of assignment –Defined standards –Contribute to course grade Outside of the course (Assessment of SLOs) –Faculty agreement of essentials = double duty –Facilitates discussion –Improve course’s effectiveness and student learning

4. Benefits of a Rubric Instructor Objective, clear, consistent evaluation tool Evaluates students’ work for course grade Use with a variety of assignments AND Evaluates grouped student work enabling teachers to gather data to evaluate the effectiveness of a course within a program and thus improve student learning (SLO assessment)

5. Benefits of a Rubric Students Roadmap for excellence on assignments Reveals “what counts”, “what is important” Clear sense of what is to be achieved Understand the grades that are earned Standards of the profession/discipline

6. Essentials of a Good Rubric Criteria must align with the SLO by identify the various characteristics of the outcome. Scoring categories specify various levels of achievement in each characteristic. Assessment task should mirror real-life challenges as appropriate. Two independent raters will arrive at a similar score for a given response based on the scoring rubric.

7. Components of a Rubric Assignment = task components of a product or performance/clear definitions of each characteristic to be assessed Evaluation = criteria standards or competencies Levels of Achievement = scale of success by letter grades, numbers, or narrative Documentation

8. Assignment = task Product –Essay –Portfolio –Final project –Shop product –Document/Map Performance –Speech –Dance/Act –Play an instrument/Sing –Skill activity –Lab procedure

9. Evaluation = criteria Elements of the assignment Evaluation criteria & standards –Demonstration of requirements –Evidence of specific progress –Feedback on competencies –Miss, meet or exceed standards

10. Achievement = scale Letter Grade A, B, C, D, F Numbers 1-3 (simple) ; 1-4; 1-5 (detailed) Narrative “beginning, developing, competent, exemplary”; many versions

11. Documentation = samples Product Performance Letter grade Number scale Narrative explanations

12. “Product” Excerpt Excel Spreadsheet Final Project Below Expectation 1 Satisfactory 2 Exemplary 3 Requirements Document Sheet Requirements document is incomplete. Many sections are missing. Descriptions are missing or not written thoroughly. Requirements document is not complete. Includes most of the assigned sections. Missing 2 or more sections or lacks accurate descriptions. Requirements document is complete. Includes all assigned sections in addition to thorough and accurate descriptions. Worksheet Design & Formatting Incomplete-missing more than two: Missing labels & values Title & subtitle Column headings Missing title merged and centered 3 or more spelling errors Somewhat complete- missing two: Pertinent labels & values Title & subtitle Column headings Title merged and centered Fewer than 2 spelling errors High level of professionalism All labels & values included Appropriate titles & subtitles added Title merged and centered No spelling errors

13. “Performance” Sample Golf Bunker Shot BeginningDevelopingCompetent StanceClosed stance Posture – bent over, stiff knees, weight even Open stance Posture – straight back, slightly bent knees, weight slightly forward Open stance Posture - straight back, slightly bent knees, weight forward Ball PositionBall positioned too far forward in the stance Ball positioned too close or too far to the golfer Ball positioned in the middle of the stance Ball positioned at a distance from the golfer that impacts the gold posture Ball positioned middle to back of the stance Ball positioned the correct distance from the golfer that allows good golf posture Swing TechniqueClosed club face Halting, shortened/too large of a swing Eyes move Erratic arc of the swing Parallel club face Jerky, but complete swing Eyes down Complete swing, but off line Open club face Smooth, rhythmic swing Eyes behind the ball “Down the target line” swing

14. “Grade” Sample Political Science Essay A = The A-range essay implies a strong argument and provides convincing specific support from the various readings. The writer demonstrates mature command of language through a variety of sentence structures, word choices, quotes or paraphrases from the readings (consistently cited correctly). Control of usage and mechanics, despite occasional flaws, contributes to the writer’s ability to communicate the purpose of the paper. The writer thoroughly understands the concepts/theories involved and through the essay can convince others of his/her viewpoints or helps make the reader aware of something completely new or original. These are powerful due to organization and creativity. B = The B-range essay shows effort and promise for the writer. It presents a thesis (argument) and often suggests a plan of development that is carried out effectively. Mastery of the readings/theories may not be fully indicated with the use of quotes or paraphrases, but the writer provides enough supporting details, makes competent use of language, and sometimes varies sentence structure. Occasional errors in usage and mechanics do not interfere with the writer’s ability to communicate the purpose of the paper. C= The C-range essay presents a thesis (argument) and often suggests a plan of development which is generally c arrived out. The writer may or may not have completed all of the required readings and utilizes generalizations or list for support. Command of the theories under consideration is weak or shaky. Sentence structure tends to be repetitious, and errors in usage and mechanics sometimes interfere with the writer’s ability to communicate the purpose of the paper. D = The D-range may present a thesis (argument); however, the plan of development is usually not carried out. This indicates the writer may have spent little time with the readings or thinking about the concepts involved. The writer provides support that tends to be sketchy and/or illogical. Sentence structure is simplistic, repetitious and occasionally awkward. Language is often inappropriate in tone or style. Errors in usage and mechanics are frequent. F = This paper presents a thesis that is vaguely worded, weakly asserted or there is no central argument present. Support, if any, tends to be rambling and superficial. Sentence structure is difficult to follow and errors in usage and mechanics interfere with the writer’s ability to communicate the purpose of the paper.

15. “Numeric” Sample Philosophy - Critical Thinking 4 Consistently does all or almost all of the following: Accurately interprets evidence, statements, graphics, questions, etc.; Identifies the salient arguments (reasons and claims) pro and con.; Thoughtfully analyzes and evaluates major alternative points of view.; Draws warranted, judicious, non-fallacious conclusions.; Justifies key results and procedures, explains assumptions and reasons.; Fair-mindedly follows where evidence and reasons lead. 3 Does most or many of the following: Accurately interprets evidence, statements, graphics, questions, etc.; Identifies relevant arguments (reasons and claims) pro and con.; Offers analyses and evaluations of obvious alternative points of view.; Draws warranted, non-fallacious conclusions.; Justifies some results or procedures, explains reasons.; Fair-mindedly follows where evidence and reasons lead. 2 Does most or many of the following: Misinterprets evidence, statements, graphics, questions, etc.; Fails to identify strong, relevant counter-arguments.; Ignores or superficially evaluates obvious alternative points of view.; Draws unwarranted or fallacious conclusions.; Justifies few results or procedures, seldom explains reasons.; Regardless of the evidence or reasons, maintains or defends views based on self-interest or preconceptions. 1 Consistently does all or almost all of the following: Offers biased interpretations of evidence, statements, graphics, questions, information, or the points of view of others.; Fails to identify or hastily dismisses strong, relevant counter-arguments.; Ignores or superficially evaluates obvious alternative points of view.; Argues using fallacious or irrelevant reasons, and unwarranted claims.; Does not justify results or procedures, nor explain reasons.; Regardless of the evidence or reasons, maintains or defends views based on self-interest or preconceptions.; Exhibits close-mindedness or hostility to reason.

16. “Narrative” Excerpt English 1/ESL 1 Final Essay Examination Sampling of Narrative Criteria A.Respond to the writing task. The paper must respond to the specific topic and the reading and must show understanding of the reading. A paper simply restating the content of the reading is not an adequate response. The paper must be at least 400 words long. B.Treat the reading fairly. The paper must avoid plagiarism by attributing the words and ideas of others to their source. Borrowed words and combinations of words must be set off by quotation marks. The paper must not misinterpret or misquote others’ words. C.Maintain a consistent style and sense of audience. The paper must not have abrupt or confusing shifts in person, voice or tense. E. Be organized logically and coherently. The paper must be organized according to a discernible plan, with the main points supporting the thesis. The reader must be able to follow the plan with ease. I.Adhere to the conventions of Standard English. The paper should have no more than minimal errors in spelling, punctuation, capitalization, apostrophe, and case, and should use standard forms of verbs, adjectives, and adverbs.

17. Recap Why a Rubric? Can be used for evaluation both within a class and for assessment of SLOs ✓ Assists instructors to precisely define course expectations across all sections ✓ Provides faculty with a tool that can be used to evaluate student work and ultimately to gather data to evaluate the effectiveness of student learning Assists students with what is needed to be successful in a course Can be used for a product or performance Has flexibility with its design