Oral Testing of Accuracy and Fluency JoAnn Miller, Editorial Macmillan miller@efltasks.net www.efltasks.net Hello, I’m JoAnn Miller. In this talk I will share with you some oral exam procedures that I developed for a private university in Mexico.

Fluency The ability to produce written and / or spoken language with ease Speak with a good but not necessarily perfect command of intonation, vocabulary and grammar Communicate ideas effectively Produce continuous speech without causing comprehension difficulties or a breakdown in communication First some definitions. Note that the accuracy/fluency distinction is valid for both oral and written production and can even be considered for the input skills of listening and reading. In this talk we will concentrate on oral production. I think the key terms in these definitions are: effective communication, continuous speech without a perfect command of the more technical aspects of oral production. Longman Dictionary of Language Teaching and Applied Linguistics

Accuracy Ability to produce grammatically correct sentences May not include the ability to speak or write fluently. Accuracy is the opposite—it stresses grammatically correct sentences and does not always include the continuous speech aspect of fluency. Our students need to be able to use both abilities in their professional lives. Since, as we will see in a few minutes, accuracy assumes time for preparation, we can think of the following experiences your students might have to face. Longman Dictionary of Language Teaching and Applied Linguistics

An example: foreign visitors Tour of factory—fluency; must be prepared for unexpected questions and explanations Formal presentation of a product—accuracy; time to plan, rehearse Imagine one of your students works in a factory in Mexico that is expecting some Japanese visitors who don’t speak Spanish. Since no one at the factory speaks Japanese and the visitors do speak English, your student has been asked to do two different activities: The first is give the guests a tour of the factory. This involves fluency since although she can plan a bit about what she will say during the tour, there will be unexpected questions from the visitors. The visitors, also being second language learners, will be tolerant of any mistakes she might make just as long as they can understand what she wants to communicate. The presentation is later in the week and allows your student to prepare. She can write out her talk, prepare Power Point slides and even rehearse in front of her friends. This previous preparation allows for accuracy and the visitors (as well as her bosses) will be expecting a more formal, “correct” performance.

What is a task? A goal-oriented activity in which learners use language to achieve a real outcome. Learners use whatever target language resources they have in order to solve a problem, do a puzzle, play a game, or share and compare experiences. I became aware of the importance of the accuracy / fluency distinction when the university where I was working adopted a task-based syllabus. Here you can see what a task is. Note that students are not limited by accuracy when doing a task..Tasks are fluency practices.

Fluency in Tasks “Learners need opportunities to process language for communicative purposes as receivers and producers. “These opportunities should be unfettered by the perceived need to conform to teacher expectations in terms of the production of specific language forms.” Tasks are activities in which students experiment with the language in order to communicate their ideas without worrying directly about accuracy. Dave Willis, “Accuracy, fluency and conformity” Challenge and Change in Language Teaching, J. Willis and D. Willis, ed. Heinemann, 1996. P. 50

Accuracy in Tasks “Whenever learners are involved in communication they are concerned with accuracy…making the best use of their language systems… “In spontaneous communication [they] have little time to reflect on the language they produce. “ If…they are given time to prepare what they have to produce, there will be a concern for formal accuracy…” However, it is recognized that accuracy is also important. Accuracy comes through preparation. Dave Willis, “Accuracy, fluency and conformity” Challenge and Change in Language Teaching, J. Willis and D. Willis, ed. Heinemann, 1996. P. 50

Speaking Fluency: “concerns the learner’s capacity to produce language in real time without undue pausing or hesitation.” Accuracy: “how well language is produced in relation to the rule system of the target language.” Therefore, in oral work we have to practice both abilities. Peter Skehan, “Second language acquisition research and task-based instruction”, Challenge and Change in Language Teaching, J. Willis and D. Willis, ed. Heinemann, 1996. P. 22.

Implications “Teachers should balance issues of fluency and accuracy depending on the specific needs of learners and the resources of time and materials for instruction.” This balance will depend on the needs of the students. Students who are planning to go into tourism will need more fluency practice. Students who are interested in translation will need more accuracy practice. Most students fall into the middle ground and will do well with practice in both skills. Miriam Eisenstein Ebsworth, “Accuracy Vs. Fluency: Which Comes First in ESL Instruction?”, ESL Magazine. 1:2, 24-26. March/April 1998.

The teaching situation Private, multi-level (high school, university, post-grad) university in Mexico (17 campuses throughout country) Institutional EFL programs Total 35,000+ students Groups of 25-30 in language lab Native and non-native teachers with varying abilities and experience I was coordinating the English language program at a large private university in Mexico. The courses were institutional—we supplied the programs, texts and exams for all students. The exams we will be seeing here were for the BA program. The students in this program have 5 hours of English courses a week: 4 in a classroom and one hour in a language lab. They usually have different teachers for the classroom and the lab. The oral exams are given in the lab. This puts less pressure on the students when practicing oral English in the classroom since they know that teacher will not be grading their production.

Grading Objective grading required as much as possible (by government, school and parents) Each course: 2 written exams in classroom (functions, structures, vocabulary, reading, writing); 2 exams in lab (listening comprehension and oral production) Since we were giving both fluency and accuracy practice in class, we felt this should be reflected in both the written and oral exams. In the written exams, besides the more traditional formats, students are also asked to produce parts of a conversation. These responses are graded based on if they communicate a logical idea or not. Minor errors in grammar are not counted, but errors in function are since it is possible for a student to make a number of grammatical errors and still communicate while an error in function usually impedes communication.

Oral Exam Fluency Accuracy Role play with teacher and another student No preparation Cues only Accuracy Role play 5 minutes+ preparation Perform for teacher 1-2 minutes maximum The oral exams are based on role plays. The accuracy exam is given first since students need preparation time. The fluency exam has no preparation. Both role plays are presented only for the teacher. The other students are either preparing their presentations or have already left. Students spend an average of 3 minutes per pair doing their role plays which are designed to be short. This allows the teacher to test the entire group of 30 in a 50 minute class period.

Oral Exam Organization Students (pairs) given role cards as they enter classroom Told to prepare—when finish exam, can leave Can use any reference or ask questions Come to the front of the class, talk only to teacher Perform Either stay together or separate for second role-play As students come into the lab they are put into pairs and given a role play card. They are told they will have AT LEAST five minutes to prepare. They are also told that as soon as they present their role plays for the teacher, they can leave. This encourages many of them to work fast. While they are preparing they can use any resource they want: textbooks, dictionaries or even ask questions. When they are ready, they come to the front of the class. They can have no notes or papers with them, so even if they wrote out their roles, they can’t read them. After they have performed the accuracy role play, the teacher gives them, either individually or in pairs, instructions for the fluency role play. They are given no preparation time.

5 point scale Accuracy Fluency 1 ---Little or no language produced. 2 --Poor vocabulary, serious mistakes in grammar, poor pronunciation. 3 --Adequate vocabulary, mistakes in grammar, adequate pronunciation. 4 --Good vocabulary, occasional errors in grammar, good pronunciation. 5 --Wide vocabulary, very few errors in grammar, very good pronunciation. Fluency 1 ---Little or no communication. 2 ---Very hesitant and brief utterances, sometimes difficult to understand. 3 ---Communicates ideas, but hesitantly and briefly 4 ---Effective communication, but does not elaborate on response. 5 ---Easy and efficient communication. Elaborates on responses. This is the 5 points scale we used for grading. It was developed from numerous popular scales and adapted to our specific needs. Students can only get 0 if they do not present the exam. Most students get between 3 and 4 points on both scales.

I. Pair Roleplay. Used to grade accuracy. (5 points) Instructions: Give each pair of SS a roleplay card. Give them at least 5 minutes to prepare their talk. Call pairs of Ss to the front. They perform without written notes. Grade them based on the following scale: [5-point scale presented here.] Note: Please take S's level into account. A Course 1 student cannot produce as much language as a Course 3 student. To get 3 points, the student should be able to use structures and vocabulary taught in the course he/she just finished. However, expect errors since the student has not fully acquired the material. To get 5 points, the student may still make a few isolated errors, but will speak much above a typical student at the same level. These are the instructions that are given to the teachers.

II. Roleplay with the teacher. Use to grade fluency. (5 points) Instructions: Choose one of the situations given below. If three people are needed in the situation, keep the pair together. If not, separate Ss and grade individually. S does not see situations. Explain the situation to the S and perform your part of the roleplay. Grade S on the following scale: [5-point scale presented here.] Note: Students should be graded considering their course level.

Student A: Imagine you are at a meeting and not in an exam Student A: Imagine you are at a meeting and not in an exam. All of your classmates are at the meeting too. Your partner doesn’t know anyone. Tell him who the people are. Student B: Imagine you are at a meeting and not in an exam. All of your classmates are at the meeting too. You don’t know anyone. Ask your partner who the people are. Student A: Tell your partner about an accident you or some member of your family had. When he/she tells you, ask some intelligent questions or make relevant comments. Student B: Tell your partner about an accident you or some member of your family had. When he/she tells you, ask some intelligent questions or make relevant comments. Student A: You are making a survey about what people think they will be able to do with telecommunications in twenty years. Ask your partner at least three questions about the topic. Student B: Your partner is making a survey. Answer his/her questions. These are some of the role plays that were used for testing accuracy. They are cut into cards. The role plays vary according to level and reflect functions practiced in the textbook. There are various cycles in use for each exam to discourage dishonesty, but in reality, since students are given preparation time, it doesn’t matter much if they get one of the cards before class since they can’t know which of the role plays they will be asked to do.

II. Roleplay with the teacher. Use to grade fluency II. Roleplay with the teacher. Use to grade fluency. (5 points) (Do not show these questions to the Ss. ) Keep students together. Ask them what they plan to do when they finish school. Then ask them to tell you the pros and cons of that job. Separate students for a moment. You are going to give one student a message for the other student. For example, ask Student A to tell Student B you are going to meet him/her after class. Then have the student pass on the message. Make the messages a little bit complicated. When you finish, give Student B messages for Student A. 3. Separate students. Tell student to imagine his girlfriend / her boyfriend is angry. Ask him / her what he / she will do. Then ask a “what if” question: What if he doesn’t believe you? What if he goes out with someone else?, etc. This is an example of the kinds of role plays used for the fluency part of the exam. The teacher can choose which role play to use to add variety and to not let the students know which they will get.

Part I is the listening comprehension exam. It is on the same page. II. Oral Exam (10 points) Role Play 1: 5 4 3 2 1 0 Role Play 2: 5 4 3 2 1 0 Total Oral points: ___/10 In order to make grading easier, this is the form used by the teachers to record the points. In conclusion, teachers and students have responded favorably to this kind of testing since they feel it allow comprehensive oral testing in a relatively friendly environment.

Thank you very much…. JoAnn Miller miller@efltasks.net miller@room20.org www.efltasks.net www.room20.org www.salondemaestros.com