MPDP 2010 Session 3: CREATING RUBRICS

What are RUBRICS? Define “rubric”. Have you ever worked with rubrics? What did you use your rubrics for? Have you ever written a rubric?

Definition Scoring rubric A set of scoring guidelines or criteria used in scoring or judging a test taker’s product, performance or response. Rubrics (descriptors) are used to make raters’ subjective judgements more reliable. Longman Dictionary of Language Teaching & Applied Linguistics

Rubrics are tools that formalize the process of evaluation. Essentially, a rubric consists of a fixed measurement scale and a set of criteria that are used to discriminate among different degrees of quality or levels of proficiency.

It’s an authentic assessment tool used to measure students’ work. It’s a scoring guide that seeks to evaluate a student’s performance based on the sum of a full range of criteria, rather than a single numeric score.

Rubrics answer the following questions: By what criteria should performance be judged? Where should we look and what should we look for to judge performance success? What does the range in the quality of performance look like? How do we determine validly, reliably, and fairly what score should be given and what that score means? How should the different levels of quality be described and distinguished from one another?

Elements you are going to evaluate: Number of chocolate chips Color Taste The chocolate chip rubric

Evaluation criteria: Delicious = 4 Good = 3 Needs Improvement = 2 Poor = 1 Now we have to write descriptors for the criteria.

The descriptors for “delicious” (what we mean by delicious = 4): Chocolate chips in every bite (number of chocolate chips) Golden Brown (color) Home-baked taste, rich, creamy (taste) Descriptors

In Pairs. Write the descriptors for: Good = 3 Needs improvement = 2 Poor = 1

4 Delicious 3 Good 2 Needs improvement 1 Poor Number of chocolate chips Chocolate chip in every bite ColorRich and creamy home- baked taste TasteGolden brown

Share your rubric with other teachers. Compare, discuss. Edit your rubric.

A typical rubric… Contains a scale of possible points to be assigned in scoring work, on a continuum of quality. High numbers usually are assigned to the best performances: Scales typically use 4, 5 or 6 as the top score, down to 1 or 0 for the lowest scores in performance assessment. Provides descriptors for each level of performance to enable more reliable and unbiased scoring.

A typical rubric… Is either holistic or analytic. If holistic, a rubric has only one general descriptor for performance as a whole. If analytic, there are multiple rubrics corresponding to each independent dimension of performance being scored.

A typical rubric… Is generic, genre specific, or task specific: If generic it can be used to judge a very broad performance, such as communication or problem solving. If genre specific, it applies to a more specific type of performance within the broad performance category (e.g. essay, speech, or narrative as forms of communication: open-ended problems or close- ended problems as kinds of problems solved) Task specific is unique to a single task.

A typical rubric… May be longitudinal. It measures progress over time toward mastery of educational objectives such that we assess developmental change in sophistication or level of performance.

It is a good idea to share rubrics with the students: Students have explicit guidelines regarding teacher’s expectations. Students can use Rubrics as a tool to develop their abilities.

Rubrics allow instructors and students to pinpoint exactly what skills within a specific task need to be worked on through further instruction. Rubrics provide descriptors for each level of performance to enable more reliable and unbiased scoring.They are "uniform and consistent."

Scoring rubrics are becoming one of the most popular forms of authentic assessment and they are used by ESL/EFL practitioners in a variety of educational contexts.

Creating a rubric When creating a rubric you have to think of: Aspects to be evaluated Evaluation criteria Descriptors for the criteria (when necessary)

Aspects to observe: Speaking Speaks in complete sentences. Takes turns appropriately when engaged in a conversation. Pronounces speech sounds and words appropriately for age and language level. Responds correctly to questions made by teacher. Uses increased vocabulary throughout the year. Uses increasingly complex oral language adequate for age and level.

Creating a rubric The first step in developing a scoring rubric is to clearly identify the qualities that need to be displayed in a student's work to demonstrate proficient performance (Brookhart, 1999). The identified qualities will form the top level or levels of scoring criteria for the scoring rubric.

After defining the criteria for the top level of performance, the evaluator's attention may be turned to defining the criteria for lowest level of performance. What type of performance would suggest a very limited understanding of the concepts that are being assessed?

The contrast between the criteria for top level performance and bottom level performance is likely to suggest appropriate criteria for middle level of performance. This approach would result in three score levels. It is better to have a few meaningful score categories then to have many score categories that are difficult or impossible to distinguish.

Evaluation Criteria Always A FrequentlyF SometimesS NeverN

Aspects to observe Speaks in complete sentences. Takes turns appropriately when engaged in a conversation. Pronounces speech sounds and words appropriately for age and language level. Responds correctly to questions made by teacher. Uses increased vocabulary throughout the year. Uses increasingly complex oral language adequate for age and level.

Descriptors Language used for descriptors should: be decided on by a group of teachers be clear, unambiguous, to the point be reader / user friendly

Weighting Aspects could have equal importance, or be discriminated: Example: Complete sentences 40% Turn taking 30% Pronunciation15% Vocabulary15%

Weighted rubrics are used to explicitly demonstrate to students and parents which criteria take precedence over others.

Now you write your own rubric. Make it one you could use in your classes. Find a partner. You are going to write a rubric to evaluate written performance. Which aspects would you observe? Workshop

SKILL: Writing ASPECTS TO BE OBSERVED: 1. Spelling and punctuation are appropriate for age and level CRITERIA: A / F / S / N DESCRIPTORS Workshop

Writes complete, grammatically correct sentences Uses vocabulary learned in this level Spelling and punctuation are appropriate for age and level AFSNAFSN X X X X Communicates ideas in every sentence and paragraph.

Dear Yukio, Hello! My name is Rafael and I am a Spanish. My teacher give me your name from a list of Japanese penfriends. I have 13 years old and I live in Zaragoza. Zaragoza is in the north of Spain and is a very beautiful city.There are four persons in my family. My father works on a bank and my mother is nurse. I have a sister, Alicia, who is six. We also a have a dog called Pluto. but he is very old and a bit estupid. I like swimming and listening music and my favourite subject is history. Do you like music too? What are your preferred groups? My teacher says that you live in Nagasaki. Is a nice place? Do you like to write to me? Please write soon. Best wishes. Rafael Adapted from “Evaluating your Students” Richmond

Keeping standards Rubrics also help standardize evaluation processes when all teachers use them. This makes the teaching and learning process one that is measurable and comparable in time.

Let me make your life easier!

Happy Teaching! Jimena Lizalde Academic Services Coordinator