1. Teaching Organization (an inductive approach) Each group has a set of sentences that used to be a paragraph. The paragraphs, when properly ordered, made an essay. Arrange your set of sentences into a paragraph that makes sense. Be ready to explain WHY you put them in the order you selected. (If you had some tough decisions, explain how you made the call – point to specific words or phrases that influenced your decision. Once we have paragraphs, we’ll arrange them into an essay, noting WHY we put them in the order we selected. Finally, we will derive some “rules” (OK, “guidelines”) for organizing texts.

2. 3-10-16 Review: Responding to Student Writing Understanding Rubrics Practice Using a Rubric Finish TAW (Assessment) Agenda: Starter: Teaching Organization Inductively activity

3. Proofreader Fixes errors (and assumes responsibility for finding errors) Editor Polishes the text; tells writers what to do to improve (and assumes some responsibility for the quality of the revised draft) Critic Evaluates quality; points out positives and negatives (and assumes responsibility for “correctness” of evaluations) Cheerleader Praises the “good stuff” (and, like the critic, assumes responsibility for “correctness” of evaluations) Roles Readers Can Play When Responding Facilitator Helps writers make their own decisions; points out potential problem areas and suggests options (and makes the writer responsible for his or her decisions) Ally Tries to help authors get by the gatekeeper (and shares some responsibility with the writer, at least for offering good advice) Gatekeeper Stops sub-standard material from passing (and is responsible to the institution for “quality control”)

4. Quality of ideas Appropriateness of the material Accuracy of content presented Organization of ideas Depth/development of ideas Likely audience reactions Stylistic issues Grammar/mechanics issues Aspects on Which to Comment: From Straub & Lunsford, 12 Readers Reading

5. Ways to Respond: Make a correction ("there" "their“) Give a command ("Move this sentence to the opening paragraph") Make a judgment *Absolute ("Awkward transition"; “Good point”) *Subjective ("I think this subject is trivial"; “I like this revision”) Offer a suggestion ("You might try to soften the tone here") Request a change ("Can you use a more precise word here?") Request additional information ("Can you give an example of x?") Ask a question *Closed ("Did you really mean to put this in passive voice?") *Leading ("How can you tie this point to the preceding one?") *Open ("What are some counter-arguments you might address?") React subjectively ("I laughed out loud when I read this line!") Give a related assignment ("Review the punctuation chapter") Acknowledge effort ("I can tell you're trying to add depth here") Offer encouragement ("I see improvement since last time") From Straub & Lunsford, 12 Readers Reading

6. More Options for Responding No Responding: Sharing Sometimes it’s enough simply to let the writer read aloud Descriptive Responding Sayback – Tell the reader what the text says to you Pointing – Point to, or identify, key words or phrases What’s Almost Said – Identify what the writing implies Structure / Voice / Point of View / Level of Abstraction / Attitude toward Reader / Language / Diction / Syntax – Identify these aspects for the writer Metaphorical Description – Describe the shape or some other feature of the text Analytical Responding Skeleton feedback – Identify reasons & support, assumptions, and the implied audience Believing – Accept everything & offer additional ideas to help build the case Doubting – Challenge everything & offer counter arguments that are not addressed Descriptive outline – Explain what the text says and does Reader-Based Responding: Movies of the Reader’s Mind Criteria-Based Responding: Compare Text to a Rubric From Elbow & Belanoff, Sharing & Responding

7. Teacher Comments on Effective and Ineffective Responses… Traits of Effective Responses Specific goals/questions/tasks Focused/targeted Positive (i.e., praise) Evaluations included suggestions Not too numerous: just a few comments per paper Maybe combine with a short conference to be sure students understand comments Train students how to read our particular comments Traits of Ineffective Responses Generic (“nice,””good,””awk”) Not fully explained Too many comments per paper Incomplete thoughts Illegible

17. Responding could be, but is not necessarily, the same as Assessing which could be, but is not necessarily, the same as Grading

18. What does a grade mean? Does an A in an Advanced Placement class mean the same as an A in a “regular” class? Does a B in Mrs. Smith’s class mean the same as a B in Mr. Jones’ class? What’s the difference between an 83 and an 85? Does a C mean “OK” or “almost failing”? Grades are not inherently meaningful; they can serve different purposes and even have different meanings in different situations. We therefore need to explain grades to invest them with meaning.

23. What IS a rubric?

24. A measuring stick against which to compare something.

25. Rubrics come in two basic varieties: Holistic (like the SAT rubrics):Analytical (like the VALUE rubrics*): *Here at The Citadel, we use VALUE rubrics for a variety of classes.

27. SCORE CONTENT/DEVELOPMENTORGANIZATIONVOICECONVENTIONS 4 Presents a clear central idea about the topic Fully develops the central idea with specific, relevant details Sustains focus on central idea throughout the writing Has a clear introduction, body, and conclusion. Provides a smooth progression of ideas throughout the writing. Minor errors in standard written English may be present. 3  Presents a central idea about the topic  Develops the central idea but details are general, or the elaboration may be uneven  Focus may shift slightly, but is generally sustained  Has an introduction, body, and conclusion.  Provides a logical progression of ideas throughout the writing.  Uses precise and/or vivid vocabulary appropriate for the topic  Phrasing is effective, not predictable or obvious  Varies sentence structure to promote rhythmic reading  Strongly aware of audience and task; tone is consistent and appropriate  Errors in standard written English may be present; however, these errors do not interfere with the writer’s meaning. 2 Central idea may be unclear Details may be sparse; more information is needed to clarify the central idea Focus may shift or be lost causing confusion for the reader  Attempts an introduction, body, and conclusion; however, one or more of these components could be weak or ineffective.  Provides a simplistic, repetitious, or somewhat random progression of ideas throughout the writing.  Uses both general and precise vocabulary  Phrasing may not be effective, and may be predictable or obvious  Some sentence variety results in reading that is somewhat rhythmic; may be mechanical  Aware of audience and task; tone is appropriate  A pattern of errors in more than one category (e.g., capitalization, spelling, punctuation, sentence formation) of standard written English is present; these errors interfere somewhat with the writer’s meaning. 1 There is no clear central idea Details are absent or confusing There is no sense of focus  Attempts an introduction, body, and conclusion; however, one or more of these components could be absent or confusing.  Presents information in a random or illogical order throughout the writing.  Uses simple vocabulary  Phrasing repetitive or confusing  There is little sentence variety; reading is monotonous  There is little awareness of audience and task; tone may be inappropriate  Frequent and serious errors in more than one category (e.g., capitalization, spelling, punctuation, sentence formation) of standard written English are present; these errors severely interfere with the writer’s meaning. B Blank Off Topic Insufficient amount of original writing to evaluate Unreadable or illegible OT IS UR

29. Basic Differences AnalyticalHolistic Score = sum of scores for individual traits Score = reader’s impression of overall quality Often Formative: intended to help identify specific areas to work on Usually Summative: intended only to measure the quality of the performance Yields more information Quicker & easier

30. Why use rubrics? To identify the target To save time To avoid bias To be transparent about grades

31. Some folks try to oversell rubrics… No, rubrics merely create the illusion of objectivity.

32. Consider the SAT rubric. It looks objective…

33. variety in sentence structure meaningful variety 6 - demonstrates meaningful variety in sentence structure variety 5 - demonstrates variety in sentence structure some variety 4 - demonstrates some variety in sentence structure 3 - lacks variety or demonstrates problems in sentence structure 2 - demonstrates frequent problems in sentence structure 1 - demonstrates severe flaws in sentence structure organization, focus, coherence, and progression of ideas well organizedclearly focused 6 - is well organized and clearly focused well organizedfocused 5 - is well organized and focused generally organizedfocused 4 - is generally organized and focused 3 - is limited in its organization or focus 2 - is poorly organized and/or focused 1 - is disorganized or unfocused … but in practice, how “objectively” different are these features?

34. What features are scored? What features are omitted? 6 Traits Writing (popular for K-12) AACU Value Rubric Written Communication SAT English Essay (for college admissions) Context & Purpose Content Development Genre / Conventions Sources & Evidence Syntax & Mechanics Development Organization Language Sentences Errors Ideas Organization Voice Sentences Conventions Word Choice

35. What features are scored? What features are omitted? 6 Traits Writing (popular for K-12) AACU Value Rubric Written Communication SAT English Essay (for college admissions) Context & Purpose Content Development Genre / Conventions Sources & Evidence Syntax & Mechanics Development Organization Language Sentences Errors Ideas Organization Voice Sentences Conventions Word Choice Each rubric looks “objective,” yet each is different. Each measures writing, but each uses different criteria.

36. Reality Check: “Quick” and “efficient” aren’t the same as accurate. If rubrics were really that quick and easy, any two people would give the same score to any performance – yet, in large scoring situations (like the SAT), even professionals get different scores regularly. They have to have frequent “calibration” sessions to maintain adequate inter-rater reliability. Rubrics aren’t as simple –or as “objective” – as they seem.

37. Criterion ACriterion BCriterion C Here’s a hypothetical rubric with three criteria: Passing score

38. Hypothetical performance with three components By comparing the performance with the rubric, we should get a score for each component. Essay: A = Thesis B = Evidence C = Mechanics Speech: A = Voice B = Gestures C = Visual aids Poster: A = Content (depth) B = Content (accuracy) C = Aesthetic appeal Book Review: A = Hook B = Content C = Conventions A B C

39. C Criterion ACriterion BCriterion C Score= 87.5 Score= 92.5 Score= 93.0 A B Passing score A B C

40. If the three criteria count 1/3 each, the score = (87.5 + 92.5 + 93.0)/3 = 91 91 1/3 1/3 1/3 Passing score

41. But what if Criterion A counts for 2/3? 2/3 1/6 1/6 (87.5*2/3) + (92.5*1/6) + (93*1/6) = 89.25 89.25 Passing score

42. An institutional rubric forces the instructor to score only those features listed; it might require the instructor to set aside his/her own professional judgment about which features should be scored.

43. Reality Check: Identifying a problem doesn’t tell a student how to fix/improve anything. It’s easy to think that the job is done when the performance is scored.

44. Reality Check: Students won’t always have rubrics. Eventually, they’ll need to learn to read an assignment so they can infer the scoring criteria. Always providing a rubric can create a dependency – so always providing a rubric can be a disservice.

45. If the problems arise not from the tool but from the user, then we need to train the users properly… Rubrics don’t cause bad pedagogy People using rubrics badly cause bad pedagogy Teachers need training in how to create and use rubrics responsibly.

46. Principles for Using Rubrics Responsibly

47. (Measure what you actually teach, not simply what’s on the rubric.) ___/60: Ideas ___/40: Organization ___/100 Total Points Use a rubric that matches your instructional goals.

48. If you start with the assignment… …find or create a rubric that matches it. …create an assignment that will elicit appropriate responses. If you start with the rubric…

49. Acknowledge the limitations of rubrics. One rubric does NOT fit all assignments. You can score ONLY what’s on the rubric. Rubrics can’t “catch” unusual features. Irony? Following the teacher’s advice? Using last week’s vocabulary words? Avoiding a problem encountered last time? Creative subheadings?

50. Remember that rubrics don’t simply measure quality; rather, they define quality. “How sweet of her to write! What lovely handwriting! ” “Did Bobby really say that??? “Organization? Diction? Development? Conventions?” (Sole criterion = done/not done) (Criteria = content, readability) (Criteria = organization, diction, development, conventions)

51. Distribute rubrics to students at the BEGINNING of the assignment.

52. Use a variety of rubrics. (One size DOES NOT fit all!) Beginning Middle End _____ Word count _____ Number of examples _____Creativity _____Showing connections ____Audience awareness ____Format ____Conventions DONENOT DONE DONE / NOT DONE

53. Principles for Using Rubrics Responsibly 1.Use a rubric that matches your instructional goals. 2.Acknowledge the limitations of rubrics. 3.Remember that rubrics don’t simply measure quality; rather, they define quality. 4.Distribute rubrics to students at the BEGINNING of the assignment. 5.Use a variety of rubrics. (One size DOES NOT fit all!)

54. Keep in mind: A rubric designed for someone else’s purposes might not be of much use in your classroom.

55. So make your own rubrics!

58. Trait 1 ________ Trait 2 ________ Trait 3 ________ Trait 4 ________ Trait 5 ________ 10 pts 9 pts 8 pts 7 pts 6 pts 5 pts 4 pts 3 pts 2 pts 1 pt YOU decide what traits to score YOU decide the value of each trait YOU create “descriptors” that describe performances at each level

59. Trait 1 title Trait 2 hook Trait 3 intro Trait 4 conclusion 7 pts main idea & preview are artful, clever, or intriguing to me 6 pts intro includes main idea & preview 5 pts title grabs my interest & makes me want to read more hook grabs my interest & makes me want to read more 4 pts title reveals some, but not too much, about topic intro includes main idea or preview, but not both 3 pts hook is mildly interesting to me conclusion artfully ties up the piece 2 pts title is dull or summarizes the topic has a separate introduction conclusion gives me a sense of closure 1 pt has a title opening line seems designed to hook the reader opening paragraph is both intro & body final paragraph is separate from body paragraphs 0 pts no titleno hookno introno conclusion Suppose, for example, that you’ve been working on writing introductions and conclusions… … so you create your own 20-point rubric for scoring only the intro and conclusion:

60. Trait 1 ________ Trait 2 ________ Trait 3 ________ Trait 4 ________ Trait 5 ________ 10 pts 9 pts 8 pts 7 pts 6 pts 5 pts 4 pts 3 pts 2 pts 1 pt YOU AND YOUR STUDENTS decide what traits to score YOU AND YOUR STUDENTS decide the value of each trait YOU AND YOUR STUDENTS create “descriptors” that describe performances at each level YOUR STUDENTS learn to score their own drafts

61. WHY am I including these particular traits (and not those other ones)? WHY am I valuing this trait at X points and that one at Y points? HOW WELL do my descriptors match what I’m actually teaching? HOW WELL can my students evaluate their own work using this rubric? Creating your own rubric forces you to REFLECT on your practice:

62. Last week, you responded to – and graded – sample papers without using a rubric. Now let’s simply grade those papers using the VALUE Written Communication rubric…

64. Reality Check Assessing does not always equal grading Rubrics are no panacea for assessment Brief interview with Maja Wilson Maja Wilson, Alfie Kohn, and others have written about some of the problems with – and even potential dangers of – using rubrics.

65. A few concerns about rubrics: They create the illusion of objectivity. (The decision to include to exclude any particular trait is a subjective one.) They force readers to set aside their professional experience and read “as if” they were somebody else. (That is, rubrics create the fiction of a single reader, although in the real world, no two readers are exactly alike.) They can’t capture any feature not specifically targeted. They can stifle students’ creativity, especially if students write for the rubric. A rubric can be a substitute for “meaningful response.” The standardization of rubrics could lead to the standardization of writing.

66. Wilson’s principles for assessment (from RR, 64-65) : Honor rhetorical purpose and effect, acknowledging that different readers read differently. Encourage new insights; allow readers to identify values in response to specific pieces of writing. Understand (and value) disagreement by using it to explore the writer’s purposes and the reader’s responses. Encourage readers to articulate their positions. Teach students how to extract clarity from disagreements. (Short version: Acknowledge the messiness of writing.)

68. Gallagher’s List of Purposes Express & reflect Inquire & explore Inform & explain Analyze & interpret Take a stand Evaluate & judge Propose a solution Seek common ground You read the text differently, depending on its purpose. You should therefore also assess the text differently, depending on its purpose.

69. Subject Text Writer CONTEXT Reader What clues in the TEXT tell us what the WRITER thinks about the READER? What clues in the TEXT tell us the writer’s PURPOSE? What clues in the TEXT tell us about the CONTEXT?

70. Gallagher on Assessment Assessment should improve writing Adopt the stance of a reader, not a grader Read through a craft lens

71. Gallagher on Assessment Assessment should improve writing Adopt the stance of a reader, not a grader Read through a craft lens Assess during process, not just at the end (Surgery is too late when the patient is already dead)

72. Gallagher’s Tenets for Teaching Editing Address needs as they arise Teach to individuals more than to the class Don’t use grammar books; make them Keep the focus narrow Teach the “big” conventions (p 150) Don’t correct too much per page Student editing is generally ineffective Track the “spelling demons” Use conferences “I am not Superman…”

73. Classroom Realities Students need to see lots of models Good models > weak models You need to write with your students Gallagher’s Commandments Remember: writers are fragile Set “improvement” as your major goal Focus on the writer, not just the paper Intervene early Conference as often as possible Give students some voice with the rubric

74. Remember, it’s ENGLISH class, not just LITERATURE class. (Yes, they need knowledge, but they also need skills.)

75. For next week, read and respond to ETC, ch. 5. Philosophy statement due.