Introduction: Teaching and Testing/Assessment Lesson One Introduction: Teaching and Testing/Assessment

Contents What is a test? Why tests? Problems of many tests Qualities of a good test Terms: measurement, test, evaluation, and assessment Relationship between measurement, test, evaluation, and assessment Relationship between test/ass.& teaching Formative and summative assessment Formal vs. informal assessment Traditional vs. alternative assessment Questions to think about Homework

What Is a Test? Unavoidable, artificial, degrading? How tests make us feel much of the time? ( A necessary “evil”, “torment”? Anything positive about tests? Definition: a method of measuring a person’s ability, knowledge, or performance in a given domain. If you hear the word test in any classroom setting, your thoughts are not likely to be positive, pleasant, or affirming. The anticipation of a test is almost always accompanied by feelings of anxiety, and self-doubt. Tests seem as unavoidable as tomorrow’s sunrise in virtually every kind of educational setting. How tests make us feel much of time? Can tests be positive experiences? Can they build a person’s confidence and become learning experiences? Can they bring out the best in students? Tests need not be degrading, artificial, anxiety-provoking experiences. This class is help you to create more authentic, intrinsically motivating assessment procedures that are appropriate for their context and designed to offer constructive feedback to your students. The definition : method- an instrument- a set of techniques, procedures, or items – that requires performance on the part of the test-takers. e.g.: multiple-choice questions with prescribed correct answers; a writing prompt with scoring rubric. An oral interview based on a question script and a checklist of expected answers Measure – some tests measure general ability, while others focus on very specific competencies or objectives. Individual ability- knowledge, or performance. Testers need to understand who the test-takers are. What is their previous experience and background? Performance, or competence. A test measures a given domain. A proficiency test, the actual performance on the test involves only a sampling of skills, that domain is overall proficiency in a language- general competence in all skills of a language. A well-constructed test is an instrument that provides an accurate measure of the test-takers’ ability within a particular domain.

What Is a Test? We want to create a test. Consistently provide accurate measures of precisely the ability in which we are interested Have a beneficial effect on teaching Be economical in terms of money and time For example: the purpose of testing in this class If you hear the word test in any classroom setting, your thoughts are not likely to be positive, pleasant, or affirming. The anticipation of a test is almost always accompanied by feelings of anxiety, and self-doubt. Tests seem as unavoidable as tomorrow’s sunrise in virtually every kind of educational setting. How tests make us feel much of time? Can tests be positive experiences? Can they build a person’s confidence and become learning experiences? Can they bring out the best in students? Tests need not be degrading, artificial, anxiety-provoking experiences. This class is help you to create more authentic, intrinsically motivating assessment procedures that are appropriate for their context and designed to offer constructive feedback to your students. The definition : method- an instrument- a set of techniques, procedures, or items – that requires performance on the part of the test-takers. e.g.: multiple-choice questions with prescribed correct answers; a writing prompt with scoring rubric. An oral interview based on a question script and a checklist of expected answers Measure – some tests measure general ability, while others focus on very specific competencies or objectives. Individual ability- knowledge, or performance. Testers need to understand who the test-takers are. What is their previous experience and background? Performance, or competence. A test measures a given domain. A proficiency test, the actual performance on the test involves only a sampling of skills, that domain is overall proficiency in a language- general competence in all skills of a language. A well-constructed test is an instrument that provides an accurate measure of the test-takers’ ability within a particular domain.

What Is the purpose of testing in language leaning? to measure language proficiency regardless of any language courses that candidates may have followed To discover how far students have achieved the objectives of a course of study To diagnose student’s strengths and weakness; to identify what they know and what they do not know. To assist placement of students by identifying the stage or part of a teaching program most appropriate for their ability. If you hear the word test in any classroom setting, your thoughts are not likely to be positive, pleasant, or affirming. The anticipation of a test is almost always accompanied by feelings of anxiety, and self-doubt. Tests seem as unavoidable as tomorrow’s sunrise in virtually every kind of educational setting. How tests make us feel much of time? Can tests be positive experiences? Can they build a person’s confidence and become learning experiences? Can they bring out the best in students? Tests need not be degrading, artificial, anxiety-provoking experiences. This class is help you to create more authentic, intrinsically motivating assessment procedures that are appropriate for their context and designed to offer constructive feedback to your students. The definition : method- an instrument- a set of techniques, procedures, or items – that requires performance on the part of the test-takers. e.g.: multiple-choice questions with prescribed correct answers; a writing prompt with scoring rubric. An oral interview based on a question script and a checklist of expected answers Measure – some tests measure general ability, while others focus on very specific competencies or objectives. Individual ability- knowledge, or performance. Testers need to understand who the test-takers are. What is their previous experience and background? Performance, or competence. A test measures a given domain. A proficiency test, the actual performance on the test involves only a sampling of skills, that domain is overall proficiency in a language- general competence in all skills of a language. A well-constructed test is an instrument that provides an accurate measure of the test-takers’ ability within a particular domain.

Why Tests? (group work) Measure student’s level; divide them into the right categories Test what they have learned Give students grades Put pressure on students; concentrating in class know whether students understand or not Reach some purposes : e.g. admission Diagnose students weakness and strength

Why Tests? 1. To gather information: Achievement of learners Selection among competitors Comparison for levels/ranking Examination/Evaluation on teaching methods Identification of problem areas 2. To reinforce learning & to motivate students

Assessment and Testing Prepared administrative procedures Identifiable times in a curriculum Time-constrained A subset of assessment 2. Assessment Ongoing process ; much wider domain Evaluated by test-takers’ performance and responses. Are testing and assessment synonymous terms? No. they are not. Assessment is a popular term in current educational practice. Tests are prepared administrative procedures that occur at identifiable times in a curriculum when learners muster all their faculties to offer peak performance, knowing that their responses are being measured and evaluated. Assessment, on the other hand, is an ongoing process that encompasses a much wider domain (fields). Whenever a student responds to a question, offers a comment, or tires out a new word or structure, the teacher subconsciously makes an assessment of the students performance. For example: written work- from a jotted-down phrases to a formal essay- is performance that ultimately is assessed by self, teacher, and possibly other students. Tests, then are a subset of assessment; they are certainly not the only form of assessment that a teacher can make. Tests can be useful devices, but they are only one among many procedures and tasks that teachers can ultimately use to assess students.

Test Traditional paper-and-pencil tests A procedure designed to get a specific sample of a person’s ability at a given time (so time-constrained, a limited sample of behavior) A measurement instrument

Assessment “a set of processes through which we make judgments about a learner’s level of skills and knowledge” (Nuna 1990) Whole situation included over time An on-going process Includes multiple samples of behavior, not just one single judgment or test

Relationship between Teaching & Testing/Assessment A partnership relationship Testing not simply follow teaching Testing should be supportive of good teaching & have a corrective influence on bad teaching Depending on purposes of the test: To reinforce learning, motivate students A means of assessing Ss’ performance But now, you might be thinking, if you make assessments every time you teach something in the classroom, does all teaching involve assessments? The answer depends on your perspective. For optimal learning to take place, students in the classroom must have the freedom to experiment, to try out their own hypotheses about language without feeling that their overall competence is being judged in terms of those trials and errors. For example, tournament tennis players must before a tournament, have the freedom to practice their skills with no implications for their final placement on that day, learners have ample opportunities to play with language in a classroom without being formally graded. Teaching sets up the practice games of language learning: the opportunities for learners to listen, think, take risks, set goals, and process feedback from the coach and they recycle through the skills that they are trying to master. At the same time, during these practice activities, teacher are indeed observing students’ performance and making various evaluations of each learner. Questions like: how did the performance compare to previous performance? Which aspects of the performance were better than others? Is the learner performing up to an expected potential? How does the performance compare to that of others in the same learning community? In the ideal classroom, all these observations feed into the way the teacher provides instruction to each student.

Relationship between Test/Assessment & Teaching Tests Teaching sets up the practice games of language learning: the opportunities for learners to listen, think, take risks, set goals and process feedback from the coach and then recycle through the skills that they are trying to master. Meanwhile, during these practice activities, teachers are indeed observing students’ performance and making various evaluation of each learner. Assessment Teaching (Brown 5)

Teacher’s Role in Testing Write better tests themselves Myth: a qualified T vs. a good test maker Put pressure on professional testers to improve their tests E.g., TWE (Test of Written Eng.) in TOEFL

Formal vs. Informal Assessment (1) Systematic, planned All tests are formal assessments, but not all formal assessment is testing. E.g., journal, portfolio = formal assessment, but are not “tests” Formal assessments are exercises or procedures specifically designed to tap into a storehouse of skills and knowledge. They are systematic and planned sampling techniques constructed to give teacher and student an appraisal of student achievement. e.g. you might use a student’s journal or portfolio of materials as a formal assessment of the attainment of certain course objects, but it is problematic to call those two procedures tests

Formal Assessment Some formal assessments provide teachers with a systematic way to evaluate how well students are progressing in a particular instructional program. For example, after completing a four- to six-week theme, teachers will want to know how well students have learned the theme skills and concepts. They may give all the students a theme test in which students read, answer questions, and write about a similar theme concept. This type of assessment allows the teacher to evaluate all the students systematically on the important skills and concepts in the theme by using real reading and writing experiences that fit with the instruction. In other situations, or for certain students, teachers might use a skills test to examine specific skills or strategies taught in a theme.

Formal vs. Informal Assessment (2) Embedded in classroom tasks To elicit performance without recording results and making fixed judgments about a student’s competence. E.g., marginal comments on papers Informal assessment can take a number of forms, starting with incidental, unplanned comments and responses, along with coaching and other impromptu feedback to the students. For example: Nice job, good work.

Informal Assessment including special activities such as group or individual projects, experiments, oral presentations, demonstrations, or performances. Some informal assessments may be drawn from typical classroom activities such as assignments, journals, essays, reports, literature discussion groups, or reading logs. Other times, it will be difficult to show student progress using actual work, so teachers will need to keep notes or checklists to record their observations from student-teacher conferences or informal classroom interactions. Sometimes informal assessment is as simple as stopping during instruction to observe or to discuss with the students how learning is progressing. Any of these types of assessment can be made more formal by specifying guidelines for what and how to do them, or they can be quite informal, letting students and teachers adjust to individual needs.

Formative & Summative Assessment (1) Formative Assessment: Evaluating Ss in the process of “forming” their competences and skills Goal: helping them to continue the growth process (i.e., future continuation or formation of learning; on-going development of their language) Key: the delivery (by the teacher) and internalization (by the student) of appropriate feedback on performance E.g., most classroom assessment All kinds of Informal assessment are formative, focusing the ongoing development of the learners’ language. E.g. a comment, suggestion, or call attention to an error or feedback offered in order to improve the learners’ language ability.

Formative & Summative Assessment (2) To measure or summarize what a S has grasped at the end of a course/unit Not necessarily point the way to future progress E.g., final exams, general proficiency exams All tests (quizzes, review tests, midterm, final) are summative.

Traditional vs. alternative assessment (1) Traditional assessment: One-shot, timed Usually decontextualized Summative Product-oriented Focus on the “right” answer

Traditional vs. alternative assessment (2) Continuous long-term assessment, untimed Contextualized communicative Formative Process-oriented Open-ended, creative answers

Questions to Think about Are all tests summative? Is it possible to convert tests into “learning experiences”? Again, is testing a necessary evil? What can we use tests for?

Homework Read Chapter One in both textbooks (Brown; Bailey) Preview: Brown chapter 3 (pp. 43-47); Bailey chapters 3 & 6. Q to think about: Why is the course called Language Assessment instead of Language Testing?

Problems of Many Tests Poor quality (cf. reliability) Fail to measure accurately what is intended to be measured (cf. validity) Not practical A harmful effect on teaching/learning (i.e., harmful backwash)

Backwash (or Washback) Definition: the effect of testing on teaching/learning (Bailey 3) Harmful (negative) backwash: e.g., a writing test with multiple-choice items Beneficial (positive) backwash: e.g., an oral test

have beneficial backwash be authentic A Good Test Should … be valid be reliable be practical have beneficial backwash be authentic To rephrase the general testing problem identified above: the basic problem is to develop tests which are valid and reliable, which have a beneficial backwash effect on teaching (where this is relevant), and which are practical.