Review: Responding to Student Writing Understanding Rubrics Finish TAW (Assessment) Agenda: Starter: Discussion of Responding to Student Work

Teacher Comments on Successful and Unsuccessful Assignments… Traits of Successful Assignments Students have a degree of choice Students are interested in the work Students have a personal connection Work is relevant to student goals Assignment is concrete & specific: *clear instructions *clear expectations Teacher provides tools (scaffolding) and feedback along the way Models of successful & unsuccessful work are provided Includes low stakes elements before high stakes performance Is appropriate for students’ ability level and confidence level Traits of Unsuccessful Assignments High stakes without adequate practice Task is artificial (i.e., meaningless) Format is unclear Some terms are undefined Work has no clear value to students Work is beyond students’ capabilities Work is outside students’ comfort zone Work is outside students’ trust zone Work is overwhelming

Proofreader Fixes errors (and assumes responsibility for finding errors) Editor Polishes the text; tells writers what to do to improve (and assumes some responsibility for the quality of the revised draft) Critic Evaluates quality; points out positives and negatives (and assumes responsibility for “correctness” of evaluations) Cheerleader Praises the “good stuff” (and, like the critic, assumes responsibility for “correctness” of evaluations) Roles Readers Can Play When Responding Facilitator Helps writers make their own decisions; points out potential problem areas and suggests options (and makes the writer responsible for his or her decisions) Ally Tries to help authors get by the gatekeeper (and shares some responsibility with the writer, at least for offering good advice) Gatekeeper Stops sub-standard material from passing (and is responsible to the institution for “quality control”)

Quality of ideas Appropriateness of the material Accuracy of content presented Organization of ideas Depth/development of ideas Likely audience reactions Stylistic issues Grammar/mechanics issues Aspects on Which to Comment: From Straub & Lunsford, 12 Readers Reading

Ways to Respond: Make a correction ("there" "their“) Give a command ("Move this sentence to the opening paragraph") Make a judgment *Absolute ("Awkward transition"; “Good point”) *Subjective ("I think this subject is trivial"; “I like this revision”) Offer a suggestion ("You might try to soften the tone here") Request a change ("Can you use a more precise word here?") Request additional information ("Can you give an example of x?") Ask a question *Closed ("Did you really mean to put this in passive voice?") *Leading ("How can you tie this point to the preceding one?") *Open ("What are some counter-arguments you might address?") React subjectively ("I laughed out loud when I read this line!") Give a related assignment ("Review the punctuation chapter") Acknowledge effort ("I can tell you're trying to add depth here") Offer encouragement ("I see improvement since last time") From Straub & Lunsford, 12 Readers Reading

More Options for Responding No Responding: Sharing Sometimes it’s enough simply to let the writer read aloud Descriptive Responding Sayback – Tell the reader what the text says to you Pointing – Point to, or identify, key words or phrases What’s Almost Said – Identify what the writing implies Structure / Voice / Point of View / Level of Abstraction / Attitude toward Reader / Language / Diction / Syntax – Identify these aspects for the writer Metaphorical Description – Describe the shape or some other feature of the text Analytical Responding Skeleton feedback – Identify reasons & support, assumptions, and the implied audience Believing – Accept everything & offer additional ideas to help build the case Doubting – Challenge everything & offer counter arguments that are not addressed Descriptive outline – Explain what the text says and does Reader-Based Responding: Movies of the Reader’s Mind Criteria-Based Responding: Compare Text to a Rubric From Elbow & Belanoff, Sharing & Responding

Teacher Comments on Effective and Ineffective Responses… Traits of Effective Responses Specific goals/questions/tasks Focused/targeted Positive (i.e., praise) Evaluations included suggestions Not too numerous: just a few comments per paper Maybe combine with a short conference to be sure students understand comments Train students how to read our particular comments Traits of Ineffective Responses Generic (“nice,””good,””awk”) Not fully explained Too many comments per paper Incomplete thoughts Illegible

Responding could be, but is not necessarily, the same as Assessing which could be, but is not necessarily, the same as Grading

What does a grade mean? Does an A in an Advanced Placement class mean the same as an A in a “regular” class? Does a B in Mrs. Smith’s class mean the same as a B in Mr. Jones’ class? What’s the difference between an 83 and an 85? Does a C mean “OK” or “almost failing”? Grades are not inherently meaningful; they can serve different purposes and even have different meanings in different situations. We therefore need to explain grades to invest them with meaning.

What IS a rubric?

Rubric = measuring stick

FailPass

FailPass

FailPass

FailPass ?

A rubric should… …list the criteria being evaluated …describe a performance at each rating level Ideally, the rubric should use similar language for each level within a category, so raters compare “apples to apples” when making judgments about that particular feature. Also ideally, students should have models of each performance level for each category.

Kinds of Rubrics AnalyticalHolistic Score = sum of scores for individual traits Score = reader’s impression of overall quality Usually Formative: intended to help the writer identify specific areas to work on Usually Summative: intended only to measure the quality of the writing Yields more information Quicker & easier BOTH can be useful

SCORE CONTENT/DEVELOPMENTORGANIZATIONVOICECONVENTIONS 4 Presents a clear central idea about the topic Fully develops the central idea with specific, relevant details Sustains focus on central idea throughout the writing Has a clear introduction, body, and conclusion. Provides a smooth progression of ideas throughout the writing. Minor errors in standard written English may be present. 3  Presents a central idea about the topic  Develops the central idea but details are general, or the elaboration may be uneven  Focus may shift slightly, but is generally sustained  Has an introduction, body, and conclusion.  Provides a logical progression of ideas throughout the writing.  Uses precise and/or vivid vocabulary appropriate for the topic  Phrasing is effective, not predictable or obvious  Varies sentence structure to promote rhythmic reading  Strongly aware of audience and task; tone is consistent and appropriate  Errors in standard written English may be present; however, these errors do not interfere with the writer’s meaning. 2 Central idea may be unclear Details may be sparse; more information is needed to clarify the central idea Focus may shift or be lost causing confusion for the reader  Attempts an introduction, body, and conclusion; however, one or more of these components could be weak or ineffective.  Provides a simplistic, repetitious, or somewhat random progression of ideas throughout the writing.  Uses both general and precise vocabulary  Phrasing may not be effective, and may be predictable or obvious  Some sentence variety results in reading that is somewhat rhythmic; may be mechanical  Aware of audience and task; tone is appropriate  A pattern of errors in more than one category (e.g., capitalization, spelling, punctuation, sentence formation) of standard written English is present; these errors interfere somewhat with the writer’s meaning. 1 There is no clear central idea Details are absent or confusing There is no sense of focus  Attempts an introduction, body, and conclusion; however, one or more of these components could be absent or confusing.  Presents information in a random or illogical order throughout the writing.  Uses simple vocabulary  Phrasing repetitive or confusing  There is little sentence variety; reading is monotonous  There is little awareness of audience and task; tone may be inappropriate  Frequent and serious errors in more than one category (e.g., capitalization, spelling, punctuation, sentence formation) of standard written English are present; these errors severely interfere with the writer’s meaning. B Blank Off Topic Insufficient amount of original writing to evaluate Unreadable or illegible OT IS UR

Why USE rubrics? Identify the target Save time Avoid bias Be transparent about grades Prepare students for statewide tests

Principles for Using Rubrics Responsibly Use a rubric that matches your instructional goals. Acknowledge the limitations of rubrics. Remember that rubrics don’t simply measure quality; rather, they define quality. Distribute rubrics to students at the BEGINNING of the assignment. Use a variety of rubrics. (One size DOES NOT fit all!)

Use a rubric that matches your instructional goals. (Measure what you teach.) ___/60: Ideas ___/40: Organization ___/100 Total Points

Acknowledge the limitations of rubrics. Irony? Wit? Using last week’s vocabulary words? Creative subheadings? Avoiding a problem encountered last time? Following the teacher’s advice?

Remember that rubrics don’t simply measure quality; rather, they define quality. “How sweet of her to write! What lovely handwriting! ” “Did Bobby really say that??? “Organization? Diction? Development? Conventions?”

Distribute rubrics to students at the BEGINNING of the assignment.

Use a variety of rubrics. (One size DOES NOT fit all!) Beginning Middle End _____ Word count _____ Number of examples _____Creativity _____Showing connections _____Fourteen lines _____Iambic pentameter _____Rhyme scheme _____Coherent stanzas _____Final couplet ____Audience awareness ____Format ____Conventions DONENOT DONE DONE / NOT DONE

Last week’s sample paper got grades from D+ to A. Let’s try scoring it with a “standard” rubric. (Yes, this is a “top-down” rubric imposed on us by the administration, so the rubric is not directly related to the assignment. Still, rubrics do get imposed on teachers this way, so you might face such a situation in your classroom.)

Assignment for sample paper Select a “hot topic” in education – an issue about which there is currently discussion and disagreement – and write a paper of words in which you explain the topic and the different views, cite at least four published articles (or editorials or book chapters) relevant to the discussion, and state and support your position on the issue. Use MLA format for citations, and include a Works Cited page.

Questions or comments so far? Let’s look more closely at a high-stakes rubric for high school students…

SCORE OF 6 An essay in this category demonstrates clear and consistent mastery, although it may have a few minor errors. A typical essay effectively and insightfully develops a point of view on the issue and demonstrates outstanding critical thinking, using clearly appropriate examples, reasons, and other evidence to support its position is well organized and clearly focused, demonstrating clear coherence and smooth progression of ideas exhibits skillful use of language, using a varied, accurate, and apt vocabulary demonstrates meaningful variety in sentence structure is free of most errors in grammar, usage, and mechanics Scoring Guide for the SAT Essay

SCORE OF 5 An essay in this category demonstrates reasonably consistent mastery, although it will have occasional errors or lapses in quality. A typical essay effectively develops a point of view on the issue and demonstrates strong critical thinking, generally using appropriate examples, reasons, and other evidence to support its position is well organized and focused, demonstrating coherence and progression of ideas exhibits facility in the use of language, using appropriate vocabulary demonstrates variety in sentence structure is generally free of most errors in grammar, usage, and mechanics Scoring Guide for the SAT Essay

SCORE OF 4 An essay in this category demonstrates adequate mastery, although it will have lapses in quality. A typical essay develops a point of view on the issue and demonstrates competent critical thinking, using adequate examples, reasons, and other evidence to support its position is generally organized and focused, demonstrating some coherence and progression of ideas exhibits adequate but inconsistent facility in the use of language, using generally appropriate vocabulary demonstrates some variety in sentence structure has some errors in grammar, usage, and mechanics Scoring Guide for the SAT Essay

SCORE OF 3 An essay in this category demonstrates developing mastery, and is marked by ONE OR MORE of the following weaknesses: develops a point of view on the issue, demonstrating some critical thinking, but may do so inconsistently or use inadequate examples, reasons, or other evidence to support its position is limited in its organization or focus, or may demonstrate some lapses in coherence or progression of ideas displays developing facility in the use of language, but sometimes uses weak vocabulary or inappropriate word choice lacks variety or demonstrates problems in sentence structure contains an accumulation of errors in grammar, usage, and mechanics Scoring Guide for the SAT Essay

SCORE OF 2 An essay in this category demonstrates little mastery, and is flawed by ONE OR MORE of the following weaknesses: develops a point of view on the issue that is vague or seriously limited, and demonstrates weak critical thinking, providing inappropriate or insufficient examples, reasons, or other evidence to support its position is poorly organized and/or focused, or demonstrates serious problems with coherence or progression of ideas displays very little facility in the use of language, using very limited vocabulary or incorrect word choice demonstrates frequent problems in sentence structure contains errors in grammar, usage, and mechanics so serious that meaning is somewhat obscured Scoring Guide for the SAT Essay

SCORE OF 1 An essay in this category demonstrates very little or no mastery, and is severely flawed by ONE OR MORE of the following weaknesses: develops no viable point of view on the issue, or provides little or no evidence to support its position is disorganized or unfocused, resulting in a disjointed or incoherent essay displays fundamental errors in vocabulary demonstrates severe flaws in sentence structure contains pervasive errors in grammar, usage, or mechanics that persistently interfere with meaning Essays not written on the essay assignment will receive a score of zero. Scoring Guide for the SAT Essay

point of view on the issue; critical thinking organization, focus, coherence, and progression of ideas use of language (vocabulary) variety in sentence structure grammar, usage, and mechanics What SAT Readers are Attending To

variety in sentence structure 6 - demonstrates meaningful variety in sentence structure 5 - demonstrates variety in sentence structure 4 - demonstrates some variety in sentence structure 3 - lacks variety or demonstrates problems in sentence structure 2 - demonstrates frequent problems in sentence structure 1 - demonstrates severe flaws in sentence structure Sample Trait at Difference Performance Levels

variety in sentence structure meaningful variety 6 - demonstrates meaningful variety in sentence structure variety 5 - demonstrates variety in sentence structure some variety 4 - demonstrates some variety in sentence structure 3 - lacks variety or demonstrates problems in sentence structure 2 - demonstrates frequent problems in sentence structure 1 - demonstrates severe flaws in sentence structure Sample Trait at Difference Performance Levels

organization, focus, coherence, and progression of ideas 6 - is well organized and clearly focused 5 - is well organized and focused 4 - is generally organized and focused 3 - is limited in its organization or focus 2 - is poorly organized and/or focused 1 - is disorganized or unfocused Sample Trait at Difference Performance Levels

organization, focus, coherence, and progression of ideas well organizedclearly focused 6 - is well organized and clearly focused well organizedfocused 5 - is well organized and focused generally organizedfocused 4 - is generally organized and focused 3 - is limited in its organization or focus 2 - is poorly organized and/or focused 1 - is disorganized or unfocused Sample Trait at Difference Performance Levels

POV & Critical ThinkingOrganization & FocusUse of LanguageSentence StructureGrammar, Usage, Mechanics Analytically, the score would be , or 22. (ave = 4.4) Holistically, should it be 4.4? 4? 5?

In mass scoring sessions (such as SAT or AP), readers “calibrate” their scores using “benchmark” papers – papers that experienced readers have agreed demonstrate a particular score. Readers discuss these benchmarks papers and “recalibrate” periodically throughout the scoring session. In such readings, papers are measured against other papers in that group: that is, some papers should get the top score and some should get the bottom score, as all the scores are relative. (Hence, a given paper could get one score one year and a different score another year.)

The moral: A rubric designed for someone else’s purposes might not be of much use in your classroom.

So make your own rubrics!

Trait 1 ________ Trait 2 ________ Trait 3 ________ Trait 4 ________ Trait 5 ________ 10 pts 9 pts 8 pts 7 pts 6 pts 5 pts 4 pts 3 pts 2 pts 1 pt YOU decide what traits to score YOU decide the value of each trait YOU create “descriptors” that describe performances at each level

Trait 1 title Trait 2 hook Trait 3 intro Trait 4 conclusion 7 pts main idea & preview are artful, clever, or intriguing to me 6 pts intro includes main idea & preview 5 pts title grabs my interest & makes me want to read more hook grabs my interest & makes me want to read more 4 pts title reveals some, but not too much, about topic intro includes main idea or preview, but not both 3 pts hook is mildly interesting to me conclusion artfully ties up the piece 2 pts title is dull or summarizes the topic has a separate introduction conclusion gives me a sense of closure 1 pt has a title opening line seems designed to hook the reader opening paragraph is both intro & body final paragraph is separate from body paragraphs 0 pts no titleno hookno introno conclusion Suppose, for example, that you’ve been working on writing introductions and conclusions… … so you create your own 20-point rubric for scoring only the intro and conclusion:

Trait 1 ________ Trait 2 ________ Trait 3 ________ Trait 4 ________ Trait 5 ________ 10 pts 9 pts 8 pts 7 pts 6 pts 5 pts 4 pts 3 pts 2 pts 1 pt YOU AND YOUR STUDENTS decide what traits to score YOU AND YOUR STUDENTS decide the value of each trait YOU AND YOUR STUDENTS create “descriptors” that describe performances at each level YOUR STUDENTS learn to score their own drafts

WHY am I including these particular traits (and not those other ones)? WHY am I valuing this trait at X points and that one at Y points? HOW WELL do my descriptors match what I’m actually teaching? HOW WELL can my students evaluate their own work using this rubric? Creating your own rubric forces you to REFLECT on your practice:

Key questions: Analytical or holistic? How many traits? How many points (total or per trait) ? Who will be using the rubric?

Reality Check Assessing does not always equal grading Rubrics are no panacea for assessment Brief interview with Maja Wilson Maja Wilson, Alfie Kohn, and others have written about some of the problems with – and even potential dangers of – using rubrics.

A few concerns about rubrics: They create the illusion of objectivity. (The decision to include to exclude any particular trait is a subjective one.) They force readers to set aside their professional experience and read “as if” they were somebody else. (That is, rubrics create the fiction of a single reader, although in the real world, no two readers are exactly alike.) They can’t capture any feature not specifically targeted. They can stifle students’ creativity, especially if students write for the rubric. A rubric can be a substitute for “meaningful response.” The standardization of rubrics could lead to the standardization of writing.

Wilson’s principles for assessment (from RR, 64-65) : Honor rhetorical purpose and effect, acknowledging that different readers read differently. Encourage new insights; allow readers to identify values in response to specific pieces of writing. Understand (and value) disagreement by using it to explore the writer’s purposes and the reader’s responses. Encourage readers to articulate their positions. Teach students how to extract clarity from disagreements. (Short version: Acknowledge the messiness of writing.)

Gallagher’s List of Purposes Express & reflect Inquire & explore Inform & explain Analyze & interpret Take a stand Evaluate & judge Propose a solution Seek common ground You read the text differently, depending on its purpose. You should therefore also assess the text differently, depending on its purpose.

Subject Text Writer CONTEXT Reader What clues in the TEXT tell us what the WRITER thinks about the READER? What clues in the TEXT tell us the writer’s PURPOSE? What clues in the TEXT tell us about the CONTEXT?

Gallagher on Assessment Assessment should improve writing Adopt the stance of a reader, not a grader Read through a craft lens

Gallagher on Assessment Assessment should improve writing Adopt the stance of a reader, not a grader Read through a craft lens Assess during process, not just at the end (Surgery is too late when the patient is already dead)

Gallagher’s Tenets for Teaching Editing Address needs as they arise Teach to individuals more than to the class Don’t use grammar books; make them Keep the focus narrow Teach the “big” conventions (p 150) Don’t correct too much per page Student editing is generally ineffective Track the “spelling demons” Use conferences “I am not Superman…”

This draft looks good. I like the way you bring in an extra source to comment on the topic of grade inflation today – years after the Staples article was published. The thesis and topic sentences are generally clear (see green highlights), and you offer a fair amount of text-based evidence. Most of the sentences make sense and are grammatically sound. I marked a few errors (blue) and a few questionable lines (yellow), so you can see there’s still room to make your work clearer and cleaner – but for this point in the semester, this draft is solid. B If you can break out of the five-paragraph theme model, you might have some good papers. You make some good points, and your writing is delightfully error- free. Further, you make sense at the paragraph level. Citation note: I marked two pairs of sentences in gray. In each case, the quote illustrates some point in the sentence that precedes it. To show this connection, you could end the first sentence with a colon; then you wouldn’t have to say “he says” or “he exclaims” at the end. (Remember, a colon says, “Here’s some additional material to illustrate what I just now said.) It makes for smoother, tighter writing. Aside from the novel/short story confusion and the accompanying format errors, my only complaint is with the format: three unrelated features strung together to “support” a fairly obvious claim (that the story “illustrates the life as a parent”). That’s not really a purpose. (WHY is he illustrating parenthood? Is he maybe trying to make a point, or perhaps show that it’s not always as simple as some folks think? I think you could probably come up with a much better thesis, but that you bailed to the easy format so you wouldn’t have to think too hard. (In short, I’m saying that I think you could turn out a really good paper if you’d get away from this middle school format.) C+ The good news is that most of your sentences are solid, and a couple of the paragraphs actually engage in analysis. The bad news, however, is that the whole first third of this draft is merely a summary of the article – useful, but WAYYY too long – and you completely fail to cite any of the quotes, most of which are too long anyway. Starting out with a factual error doesn’t help. (The scene, not the whole book, takes place at a pool.) And using long quotes suggests that you don’t have much of your own to say. When you do have something to say, it’s pretty good. The paragraph that begins with “Another important lesion” (which, of course, should be “lesson,” not “lesion”) is actually pretty solid. Your writing is smooth and easy to follow, and you have something reasonable to say. If the whole paper were that good, it would get a much higher grade. I recommend that you do enough invention to give you enough to say to allow you to cut the extended summary and the overly long quotes; once you do that, you ought to have some good work. But this draft isn’t there yet. C- End Comments from papers I recently graded

Classroom Realities Students need to see lots of models Good models > weak models You need to write with your students Gallagher’s Commandments Remember: writers are fragile Set “improvement” as your major goal Focus on the writer, not just the paper Intervene early Conference as often as possible Give students some voice with the rubric

Remember, it’s ENGLISH class, not just LITERATURE class. (Yes, they need knowledge, but they also need skills.)

For next week, read and respond to ETC, ch. 5. Philosophy statement due.