Presentation on theme: "Holistic Scoring Using ACPS Measurement Topic Rubrics"— Presentation transcript:
1 Holistic Scoring Using ACPS Measurement Topic Rubrics Curriculum Implementation Modules( )Session Four: December 2011Alexandria City Public Schools
2 Essential QuestionsHow can ACPS educators collaboratively score student transfer task products using ACPS measurement topic rubrics?How can we get students to become involved in self-scoring and peer scoring using these rubrics?
3 Session Objectives By the end of this workshop, you should be able to: Build inter-rater reliability when collaboratively scoring student transfer task products using ACPS measurement topic rubrics.Promote student self-reflection and self-regulation using rubrics as part of formative and summative assessment.
4 Our Agenda at a GlanceWarm-Up Activity: Participants Share Their Students’ Performance on Transfer Tasks and Related Balanced Assessment StrategiesModeling & Discussion: An Introduction to the Concepts of Measurement Topics, ACPS Measurement Topic Rubrics, and Building Inter-Rater Reliability During the Scoring ProcessParticipant Analysis of an ACPS Curriculum Guide Transfer Task: How Can You Use Measurement Topic Rubrics to Evaluate Student Achievement on This Task?A Holistic Scoring Committee Simulation: Participants Form Small Scoring Teams and Build Consensus About the Range of “4, 3, 2, and 1” Score Points in a Sample Set of Student WritingsPlanning Opportunity: Developing an Action Plan for an In-House Scoring Committee
5 Participant Debriefing Think: How did your students do in responding to transfer tasks during the past months? (What were your success stories? What challenges did you face?)Pair: Compare your experiences with a partner.Share: Pairs share a “headline” (25 words or less) of their experiences.
6 The Big Ideas of Balanced Assessment Part I: The Importance of Balanced Assessment: diagnostic, formative, and summative forms of assessment; rapid/on-the-spot feedback; standards driven; metacognitive/student self-regulation; differentiationPart II: Diagnostic/Pre-Assessment and Formative Assessment: providing on-the-spot, criterion-based feedback to students on a daily basis; aligning assessment with teaching-learning; focusing on clear learning targets; criterion-based feedback; “closing the gap”Part III (Module Four): Lesson Planning Using Balanced Assessment to Prepare Students for Unit Transfer Tasks: using assessments that promote understanding; emphasizing oral and written language; using a range of question types and strategies; combining constructed-response test items with reflective assessments, academic prompts, and culminating projects; scoring transfer tasks
7 Part ThreePreparing Students for Transfer Tasks and Scoring Transfer Task Work Products
8 The Balanced, Photo-Album Approach to Assessment (pp. 5-6) 1. Complete the questionnaire on pages 5-6, “Creating a Photo Album, Not a Snapshot, of Assessment Results.”2. To what extent do you individually address each of the 10 identified components of a balanced, photo- album approach to assessment?
9 Based upon Your Self-Assessment, Consider (Part I)… Do you select the appropriate assessment tool or process to assess each desired result?Do you use a range of assessment tools, rather than just tests and quizzes?Do you strive for a photo album, not a snapshot, of student performance data?Does your photo album provide a full portrait of what your students know, do, and understand relative to your desired results?
10 Based upon Your Self-Assessment, Consider (Part II)… Do you make use of…Tests and quizzes that include constructed-response items?Reflective assessments (reflective journals, think logs, peer response groups, interviews)?Academic prompts with a FAT-P (audience, format, topic, purpose) clearly stated?Culminating performance assessment tasks and projects?
11 Constructed-Response Test Items (Page 7) Require some form of performance by the student within the testing situation.Involve students in demonstrations of understanding, not just knowledge-recall learning.Are often written, but can be differentiated to allow for alternative approaches (e.g., for younger students).Can involve some form of choice by the learner.
12 Sample Constructed- Response Test Items (Page 7) 1. Draw a picture to explain how a bill becomes law.Examine the following political cartoons. Write a one-paragraph analysis of how they reflect conflicting perspectives on the Vietnam War.3. Assume that you are one of the surviving characters at the conclusion of Romeo and Juliet. Write a letter to a relative explaining the major causes leading to the demise of the two protagonists.4. Observe the following videotape, which highlights elements of a local eco-system. Describe your observations and conclusions about the health of that system.5. Examine other examples on page 7.
13 ActivityWrite a sample test or quiz item that requires a “constructed” (rather than “selected”) response from students about some aspect of the knowledge, skills, and/or understandings students will need for success on the transfer task you selected.
14 Formal and Informal Reflective Assessments (pp. 8-10) Encourage students to internalize and apply to themselves and peers significant evaluation standards and criteria.Engage students in self-evaluation and meta-cognitive processing.Ensure that all learners are becoming self-monitoring and are “owning” the evaluation criteria.Encourage active feedback and adjustment.
15 Sample Reflective Assessment Activities (pp. 8-10) 1. Reflective Journal Entries: How well do you understand this passage? What are the main ideas from this lesson? What did this material mean to you?2. Think Logs: How would you describe the process of classification? How has your approach to problem-solving changed during this unit?3. Self-Evaluations: Based upon our evaluation criteria, what grade would you give yourself? Why?4. Peer Response Group Activities: What can you praise about the work? What questions can you pose? What suggestions can you make for polishing the product?5. Interviews: Tell me about your perceptions of this project. What do you consider to be your strengths and areas in need of improvement?
16 ActivityThink about the transfer task for which you are planning. Create a reflective journal entry and a think log entry for your students related to this task.
17 The Academic Prompt (pp. 11-12) A structured performance task that elicits the student’s creation of a controlled performance or product.These performances and products should align with criteria expressed in a scoring guide or rubric.Successful prompts articulate a format, audience, topic/content focus, and purpose.
18 A Sample Academic Prompt with a FAT-P Think about a time when you were surprised (topic). Write a letter (format) to a friend (audience) in which you describe that experience. Use a logical narrative sequence with concrete sensory details to help your friend understand what this event was like and how you experienced it (purpose).
19 ActivityThink about an important piece of declarative or procedural knowledge required by the task for which you are planning. Create a sample academic prompt that embodies each of the FAT-P elements (format, audience, topic, purpose) aligned with that targeted knowledge.
20 Elements of an Effective Performance Task and Culminating Project G=real-world goalsR=real-world role(s)A=real-world audienceS=real-world situationP=real-world products and performancesS=standards for acceptable performance
21 A Sample G.R.A.S.P.S. (Elementary) You and your classmates have been asked to become an historian! Each of you will research the history of a major person or event in the history of Alexandria. Then, you will write and illustrate a report that tells what you have learned, including why the person or event is important, how they influenced our city, and why we should all know about your subject. When you’ve finished, we’ll publish our reports in a class anthology (which we’ll donate to the Alexandria Historical Society). Your reports and illustrations should be accurate, complete, clearly written, and interesting for your readers.
22 A Sample G.R.A.S.P.S. (Secondary) You are a member of a team of scientists investigating deforestation of the Amazon rain forest. You are responsible for gathering scientific data (including such visual evidence as photographs) and producing a scientific report in which you summarize current conditions, possible future trends, and their implications for both the Amazon itself and its broader influence on our planet. Your report, which you will present to a United Nations sub-committee, should include an action plan that is detailed and clear with well supported recom-mendations for addressing the issues you raise.
23 Distinguishing Between an Academic Prompt and a Culminating Performance Task and Project In designing performance tasks, we need to ask ourselves: What is the level of independent transfer students are expected to demonstrate?If students are still in the area of “guided practice,” an academic prompt may be more appropriate; if students are expected to demonstrate independent transfer and a high level of conceptual understanding, a culminating project or authentic performance task (cornerstone performance) may be most appropriate.Review the performance task samples on pages of your handout. What are the major differences between these academic prompts and projects?
24 A Reflective Question for Your Consideration… Is the transfer task for which you are planning closer in format to an academic prompt or a G.R.A.S.P.S. culminating project? Why?
26 Evaluating Academic Prompts and G.R.A.S.P.S. Culminating Projects Each of the transfer tasks in the ACPS curriculum guides has two or more accompanying scoring rubrics.Examine one or more of the rubrics in the guides—and use the following slides (and the materials on pages 16-18) to identify the type of rubric included in your curriculum guide(s).Also, consider when you might choose to use a modified holistic scoring rubric, an analytical rubric, and/or an analytic scoring guide.
27 Sample Modified Holistic Scoring Rubric (P. 16) 3=All data are accurately represented on the graph. All parts of the graph are correctly labeled. The graph contains a title that clearly tells what the data show. The graph is very neat and easy to read.2=Data are accurately represented on the graph or the graph contains only minor errors. All parts of the graph are correctly labeled or the graph contains minor inaccuracies. The graph contains a title that generally tells what the data show. The graph is generally neat and readable.1=The data are inaccurately represented, contain major errors or are missing. Only some parts of the graph are correctly labeled, or labels are missing. The title does not reflect what the data show, or the title is missing. The graph is sloppy and difficult to read.
28 The Analytic-Trait Rubric (P. 17) TraitsUnderstandingPerformance or Performance QualityScaleWeights: percent35 percent4Shows a sophisticated understanding of relevant ideas and processes…The performance or product is highly effective…3Shows a solid understanding of the relevant ideas and processes…The performance or product is effective…2Shows a somewhat naïve or limited understanding of relevant ideas or processes…The performance or product is somewhat effective…1Shows little apparent understanding of the relevant ideas and processes…The performance or product is ineffective.
29 Analytic Scoring Guide (P. 18) 50%=Content: Clearly-presented thesis statement with fully-developed supporting ideas and balanced evidence to make a compelling and convincing argument.25%=Organization: Consistent support of thesis statement with all ideas and supporting evidence aligned with the controlling ideas of the composition. Consistent attention to the use of transitional expressions and other techniques to ensure coherence and clarity.25%=Editing: Elimination of major grammar and usage errors with clear attention to correct syntax and sentence variety.
30 A Few Key Points About ACPS Measurement Topic Rubrics Every transfer task has one or more measurement topic rubrics.These rubrics are holistic in design and reflect one of the K-12 measurement topics associated with the content area being taught, learned, and assessed.Teachers are encouraged to work with students to identify task-specific “bullet” descriptors for the “4” and “3” score points.Ideally, students will use these rubrics and task-specific performance descriptors to guide their work as they complete the transfer task.
31 Forming a Holistic Scoring Team Ideally, teachers responsible for the same grade level and unit transfer tasks will establish consensus about a set of “anchor” papers before scoring their own student work products.The group should assemble (e.g., during a common team or grade level planning time) and review a set of student work products for the unit transfer task (e.g., 10 or more samples).The group should first discuss the rubrics for the task and what each team member agrees they will look for while scoring.Then, team members individually score the sample papers.If a discrepancy exists (i.e., a particular paper elicits a two-point or higher spread of scores), the team must come to resolution/consensus about the score for that paper.
32 A Scoring Team Simulation Form table teams of three-four members.Review the assigned (or self-assigned) scoring rubrics.Individually score each paper.Determine if any papers require resolution/consensus building.If time permits, reflect on “outlier” papers, i.e., those that may be unusual or potentially troublesome for scoring.
33 Designing Assessments for Your Lesson Plan Consider the types of assessments and scoring tools we’ve addressed in this module.How can you use one or more of them as part of your lesson design? How will these assessment tools help your students achieve success on your unit’s transfer task?Before you begin to work on your individual lesson, do you have any questions or ideas you would like to share?
34 Preview of Coming Attractions: Module Five Our next module will deal with the issue of language acquisition, i.e., strategies for helping all learners (esp. ELL students) engage in effective reading, writing, speaking, and listening within your content area.For next time, come ready to share your experiences with scoring unit transfer tasks.Also, be thinking about how you have involved your students in self-monitoring and self-assessment.