1 Lesson One Introduction: Teaching and Testing/Assessment

2 Contents What is a test? What is a test? What is a test? What is a test? Why tests? Why tests? Why tests? Why tests? Problems of many tests Problems of many tests Problems of many tests Problems of many tests Qualities of a good test Qualities of a good test Qualities of a good test Qualities of a good test Terms: measurement, test, evaluation, and assessment Terms: measurement, test, evaluation, and assessmentmeasurementtestevaluation assessmentmeasurementtestevaluation assessment Relationship between measurement, test, evaluation, and assessment Relationship between measurement, test, evaluation, and assessment Relationship between measurement, test, evaluation, and assessment Relationship between measurement, test, evaluation, and assessment Relationship between test/ass.& teaching Relationship between test/ass.& teaching Relationship between test/ass.& teaching Relationship between test/ass.& teaching Formative and summative assessment Formative and summative assessment Formative and summative assessment Formative and summative assessment Formal vs. informal assessment Formal vs. informal assessment Formal vs. informal assessment Formal vs. informal assessment Traditional vs. alternative assessment Traditional vs. alternative assessment Traditional vs. alternative assessment Traditional vs. alternative assessment Questions to think about Questions to think about Questions to think about Questions to think about Homework Homework Homework

3 What Is a Test? Unavoidable? Unavoidable? How tests make us feel much of the time? (sample test) How tests make us feel much of the time? (sample test)sample testsample test –A necessary “ evil ” ? –Anything positive about tests? Definition: a method of measuring a person ’ s ability, knowledge, or performance in a given domain. Definition: a method of measuring a person ’ s ability, knowledge, or performance in a given domain.

4 Sample Test Circle the correct answer. You have 3 minutes to complete this test. 1. polygene A. the first stratum of lower-order protozoa containing multiple gene B. a combination of two or more plastics to produce a highly durable material. C. one of a set of cooperating genes, each producing a small quantitative effect. D. any of a number of multicellular chromosomes.

5 Why Tests? 1. To gather information: Achievement of learners Achievement of learners Selection among competitors Selection among competitors Comparison for levels/ranking Comparison for levels/ranking Examination/Evaluation on teaching methods Examination/Evaluation on teaching methods Identification of problem areas Identification of problem areas 2. To reinforce learning & to motivate students

6 Problems of Many Tests Poor quality (cf. reliability) Poor quality (cf. reliability) Fail to measure accurately what is intended to be measured (cf. validity) Fail to measure accurately what is intended to be measured (cf. validity) Not practical Not practical A harmful effect on teaching/learning A harmful effect on teaching/learning (i.e., harmful backwash) backwash

7 Backwash ( or Washback) Definition: the effect of testing on teaching/learning (Bailey 3) Definition: the effect of testing on teaching/learning (Bailey 3) Harmful (negative) backwash: Harmful (negative) backwash: –e.g., a writing test with multiple-choice items Beneficial (positive) backwash: Beneficial (positive) backwash: –e.g., an oral test

8 A Good Test Should … be valid be valid be reliable be reliable be practical be practical have beneficial backwash have beneficial backwash be authentic be authentic

9 Measurement Quantifies the characteristics (both physical and mental) of persons Quantifies the characteristics (both physical and mental) of persons –e.g.: weight, height, motivation, aptitude, intelligence Involves both tests and non-tests: test scores, ratings Involves both tests and non-tests: test scores, ratings

10 Test Traditional paper-and-pencil tests Traditional paper-and-pencil tests A procedure designed to get a specific sample of a person ’ s ability at a given time (so time-constrained, a limited sample of behavior) A procedure designed to get a specific sample of a person ’ s ability at a given time (so time-constrained, a limited sample of behavior) A measurement instrument A measurement instrument

11 Evaluation Quantitative Quantitative –Numbers involved; e.g., scores Qualitative Qualitative –Analyze data; e.g., letters of reference Systematic gathering of information for decision making Systematic gathering of information for decision making Determination of adequacy Determination of adequacy

12 Assessment “ a set of processes through which we make judgments about a learner ’ s level of skills and knowledge ” (Nuna 1990) “ a set of processes through which we make judgments about a learner ’ s level of skills and knowledge ” (Nuna 1990) Whole situation included over time Whole situation included over time An on-going process An on-going process Includes multiple samples of behavior, not just one single judgment or test Includes multiple samples of behavior, not just one single judgment or test

13 Assessment Evaluation Measurement (test) Measurement (non-test)

14 Evaluation, Measurement & Test Not all measures are tests, not all tests are evaluative, and not all evaluation involves either measurement or tests. Not all measures are tests, not all tests are evaluative, and not all evaluation involves either measurement or tests.

15 Relationship between Teaching & Testing/Assessment A partnership relationship A partnership relationship Testing not simply follow teaching Testing not simply follow teaching Testing should be supportive of good teaching & have a corrective influence on bad teaching Testing should be supportive of good teaching & have a corrective influence on bad teaching Depending on purposes of the test: Depending on purposes of the test: –To reinforce learning, motivate students –A mans of assessing Ss ’ performance

16 Relationship between Test/Assessment & Teaching Teaching Assessment Tests (Brown 5)

17 Teacher ’ s Role in Testing Write better tests themselves Write better tests themselves –Myth: a qualified T vs. a good test maker Put pressure on professional testers to improve their tests Put pressure on professional testers to improve their tests –E.g., TWE (Test of Written Eng.) in TOEFL

18 Formative & Summative Assessment (1) Formative Assessment: Formative Assessment: –Evaluating Ss in the process of “ forming ” their competences and skills –Goal: helping them to continue the growth process (i.e., future continuation or formation of learning; on-going development of their language) –Key: the delivery (by the teacher) and internalization (by the student) of appropriate feedback on performance –E.g., most classroom assessment

19 Formative & Summative Assessment (2) Summative assessment: Summative assessment: –To measure or summarize what a S has grasped at the end of a course/unit –Not necessarily point the way to future progress –E.g., final exams, general proficiency exams

20 Formal vs. Informal Assessment (1) Formal assessment: Formal assessment: –Systematic, planned –All tests are formal assessments, but not all formal assessment is testing. E.g., journal, portfolio = formal assessment, but are not “ tests ” E.g., journal, portfolio = formal assessment, but are not “ tests ” tests

21 Formal vs. Informal Assessment (2) Informal assessment: Informal assessment: –Embedded in classroom tasks –To elicit performance without recording results and making fixed judgments about a student ’ s competence. –E.g., marginal comments on papers

22 Traditional vs. alternative assessment (1) Traditional assessment: Traditional assessment: –One-shot, timed –Usually decontextualized –Summative –Product-oriented –Focus on the “ right ” answer

23 Traditional vs. alternative assessment (2) Alternative assessment: Alternative assessment: –Continuous long-term assessment, untimed –Cintextualized communicative –Formative –Process-oriented –Open-ended, creative answers

24 Questions to Think about Are all tests summative? Are all tests summative? Is it possible to convert tests into “ learning experiences ” ? Is it possible to convert tests into “ learning experiences ” ? Again, is testing a necessary evil? Again, is testing a necessary evil? What can we use tests for? What can we use tests for?

25 Homework Read Chapter One in both textbooks (Brown; Bailey) Read Chapter One in both textbooks (Brown; Bailey) Preview: Brown chapter 3 (pp ); Bailey chapters 3 & 6. Preview: Brown chapter 3 (pp ); Bailey chapters 3 & 6. Q to think about: Q to think about: –Why is the course called Language Assessment instead of Language Testing?