1. Tom Thompson Professor of English, The Citadel Director, Lowcountry Writing Project tom.thompson@citadel.edu 843-953-1418 Using Rubrics Responsibly

2. What are rubrics? Why use rubrics? Problems with rubrics Responsible usage Using Rubrics Responsibly

7. Rubric = measuring stick; a standard against which to compare something Like a measuring stick, a rubric is merely a tool. What is a rubric?

8. FailPass

9. FailPass

10. FailPass

11. FailPass ?

13. SCORE CONTENT/DEVELOPMENTORGANIZATIONVOICECONVENTIONS 4 Presents a clear central idea about the topic Fully develops the central idea with specific, relevant details Sustains focus on central idea throughout the writing Has a clear introduction, body, and conclusion. Provides a smooth progression of ideas throughout the writing. Minor errors in standard written English may be present. 3  Presents a central idea about the topic  Develops the central idea but details are general, or the elaboration may be uneven  Focus may shift slightly, but is generally sustained  Has an introduction, body, and conclusion.  Provides a logical progression of ideas throughout the writing.  Uses precise and/or vivid vocabulary appropriate for the topic  Phrasing is effective, not predictable or obvious  Varies sentence structure to promote rhythmic reading  Strongly aware of audience and task; tone is consistent and appropriate  Errors in standard written English may be present; however, these errors do not interfere with the writer’s meaning. 2 Central idea may be unclear Details may be sparse; more information is needed to clarify the central idea Focus may shift or be lost causing confusion for the reader  Attempts an introduction, body, and conclusion; however, one or more of these components could be weak or ineffective.  Provides a simplistic, repetitious, or somewhat random progression of ideas throughout the writing.  Uses both general and precise vocabulary  Phrasing may not be effective, and may be predictable or obvious  Some sentence variety results in reading that is somewhat rhythmic; may be mechanical  Aware of audience and task; tone is appropriate  A pattern of errors in more than one category (e.g., capitalization, spelling, punctuation, sentence formation) of standard written English is present; these errors interfere somewhat with the writer’s meaning. 1 There is no clear central idea Details are absent or confusing There is no sense of focus  Attempts an introduction, body, and conclusion; however, one or more of these components could be absent or confusing.  Presents information in a random or illogical order throughout the writing.  Uses simple vocabulary  Phrasing repetitive or confusing  There is little sentence variety; reading is monotonous  There is little awareness of audience and task; tone may be inappropriate  Frequent and serious errors in more than one category (e.g., capitalization, spelling, punctuation, sentence formation) of standard written English are present; these errors severely interfere with the writer’s meaning. B Blank Off Topic Insufficient amount of original writing to evaluate Unreadable or illegible OT IS UR

17. A rubric should… …list the criteria being evaluated …describe a performance at each rating level The rubric should use similar language for each level within a category, so raters compare “apples to apples” when making judgments about that particular feature. Ideally, students should have models of each performance level for each category. BOA Marching Competition Level 1Level 2Level 3Level 4Level 5 Music Performance, Individual description of performance at this level Music Performance, Ensemble description of performance at this level Visual Performance, Individual description of performance at this level Visual Performance, Ensemble description of performance at this level Visual General Effect description of performance at this level Music General Effect description of performance at this level

18. Kinds of Rubrics AnalyticalHolistic Score = sum of scores for individual traits Score = reader’s impression of overall quality Often Formative: intended to help identify specific areas to work on Usually Summative: intended only to measure the quality of the performance Yields more information Quicker & easier

19. Development Organization Language Sentences Errors These factors, taken as a group, yield a single score.

20. Development Organization Language Sentences Errors These factors, taken as a group, yield a single score.

21. Score = 4 + 2 + 4 + 3 + 3 = 16Score = 4 + 4 + 3 + 1 + 4 = 16

22. Rubric = Checklist Sonnet Scoring Guide Fourteen lines - ___ (20) Iambic pentameter - ___ (20) Rhyme scheme - ___ (20) Coherent stanzas - ___ (20) Final couplet - ___ (20) Total = ___ (100) PowerPoint Presentation Content All key points addressed___ of 33 pts Accurate Organization___ of 33 pts Beginning, Middle, End Logical progression of ideas Other___ of 34 pts Required # of slides Required # of sources Includes animation Score =___ of 100 pts

23. Why use rubrics? To identify the target To save time To avoid bias To be transparent about grades

25. Reader Text CONTEXT Writer Subject

26. DCBA

27. Read the sample student paper (“Rules of Engagement”) and, using the rubric, score it for each of the five categories.

29. Problems with rubrics They create the ILLUSION of objectivity They require readers to set aside their expertise ? They can’t account for any feature not specifically listed

31. Content Development Uses appropriate, relevant, and compelling content to illustrate mastery of the subject, conveying the writer’s understanding, and shaping the whole work Uses appropriate, relevant and compelling content to explore ideas within the context of the discipline and shape the whole work Uses appropriate and relevant content to develop and explore ideas through most of the work Uses appropriate and relevant content to develop simple ideas in some parts of the work 4 - 3 - 2 - 1 -

32. Content Development Uses appropriate, relevant, and compelling content to illustrate mastery of the subject, conveying the writer’s understanding, and shaping the whole work Uses appropriate, relevant and compelling content to explore ideas within the context of the discipline and shape the whole work Uses appropriate and relevant content to develop and explore ideas through most of the work Uses appropriate and relevant content to develop simple ideas in some parts of the work 4 - 3 - 2 - 1 -

33. Content Development Uses appropriate, relevant, and compelling content to illustrate mastery of the subject, conveying the writer’s understanding, and shaping the whole work Uses appropriate, relevant and compelling content to explore ideas within the context of the discipline and shape the whole work Uses appropriate and relevant content to develop and explore ideas through most of the work Uses appropriate and relevant content to develop simple ideas in some parts of the work 4 - 3 - 2 - 1 -

34. Uses appropriate, relevant, and compelling content to illustrate mastery of the subject, conveying the writer’s understanding, and shaping the whole work Uses appropriate, relevant and compelling content to explore ideas within the context of the discipline and shape the whole work Uses appropriate and relevant content to develop and explore ideas through most of the work Uses appropriate and relevant content to develop simple ideas in some parts of the work 4 - 3 - 2 - 1 - Content Development

36. variety in sentence structure meaningful variety 6 - demonstrates meaningful variety in sentence structure variety 5 - demonstrates variety in sentence structure some variety 4 - demonstrates some variety in sentence structure 3 - lacks variety or demonstrates problems in sentence structure 2 - demonstrates frequent problems in sentence structure 1 - demonstrates severe flaws in sentence structure organization, focus, coherence, and progression of ideas well organizedclearly focused 6 - is well organized and clearly focused well organizedfocused 5 - is well organized and focused generally organizedfocused 4 - is generally organized and focused 3 - is limited in its organization or focus 2 - is poorly organized and/or focused 1 - is disorganized or unfocused

37. What features are scored? What features are omitted? 6 Traits Writing (popular for K-12) AACU Value Rubric Written Communication SAT English Essay (for college admissions) Context & Purpose Content Development Genre / Conventions Sources & Evidence Syntax & Mechanics Development Organization Language Sentences Errors Ideas Organization Voice Sentences Conventions Word Choice

38. What features are scored? What features are omitted? 6 Traits Writing (popular for K-12) AACU Value Rubric Written Communication SAT English Essay (for college admissions) Context & Purpose Content Development Genre / Conventions Sources & Evidence Syntax & Mechanics Development Organization Language Sentences Errors Ideas Organization Voice Sentences Conventions Word Choice Each rubric looks “objective,” yet each is different. Each measures writing, but each uses different criteria.

39. ? Describe an ethical dilemma… Explain & apply the four dilemma paradigms… Explain & apply the three resolution principles… Identify the paradigms and principles that most closely fit the dilemma; state the implications of, and objections to, those paradigms and principles; defend against the objections; and explain how values influenced the decision. Student can recognize ethical issues when presented in a complex, multilayered (gray) context AND can recognize cross-relationships among the issues.

40. Irony? Wit? Allusions to current news events? Creative approach?

41. To get reliable scores, raters must be trained until they achieve high inter-rater reliability.

42. Principles for Using Rubrics Responsibly

43. Use a rubric that matches your instructional goals. If you start with the assignment… …find or create a rubric that matches it. If you start with the rubric… …create an assignment that will elicit appropriate responses.

44. Remember that rubrics don’t simply measure quality; rather, they define quality. ExcellentAverageWeak Organization Development Accuracy Grammar Mechanics

45. Acknowledge the limitations of rubrics. One rubric does NOT fit all assignments. You can score ONLY what’s on the rubric. Rubrics can’t “catch” unusual features. Rubrics aren’t objective; the decision to include or exclude any given criterion is subjective. For mass scorings, readers must be trained.

46. Distribute rubrics to students at the BEGINNING of the assignment.

47. Keep in mind: A rubric designed for someone else’s purposes might not be of much use in your classroom.

48. So make your own rubrics!

50. YOU decide what traits to score YOU decide the value of each trait YOU create “descriptors” that describe performances at each level Trait 1 ________ Trait 2 ________ Trait 3 ________ Trait 4 ________ Trait 5 ________ 10 pts 9 pts 8 pts 7 pts 6 pts 5 pts 4 pts 3 pts 2 pts 1 pt

51. Trait 1 ________ Trait 2 ________ Trait 3 ________ Trait 4 ________ Trait 5 ________ 10 pts 9 pts 8 pts 7 pts 6 pts 5 pts 4 pts 3 pts 2 pts 1 pt YOU AND YOUR STUDENTS decide what traits to score YOU AND YOUR STUDENTS decide the value of each trait YOU AND YOUR STUDENTS create “descriptors” that describe performances at each level YOUR STUDENTS learn to score their own drafts

52. WHY am I including these particular traits (and not those other ones)? WHY am I valuing this trait at X points and that one at Y points? HOW WELL do my descriptors match what I’m actually teaching? HOW WELL can my students evaluate their own work using this rubric? Creating your own rubric forces you to REFLECT on your practice:

53. Key questions: Holistic or analytical? How many traits? How many points (total or per trait) ? Who will be using the rubric?

55. Describe a way you use (or could use) a rubric more effectively and responsibly in your class. Be ready to read your description aloud.

56. Questions?

57. Tom Thompson Professor of English, The Citadel Director, Lowcountry Writing Project tom.thompson@citadel.edu 843-953-1418 Contact information, one more time: