Testing 09

Problems and Directions Past experience with language help us come to the conclusion that we should keep a balance between linguistic and psychometric considerations. Shift from psychometric-structures to psychometric-comunicative

Authenticity: communicative approach in testing Two approaches in testing: discrete-point vs. integrative Discrete-point: emphasize on structure, lexicon Integrative: emphasize on communicative effect

Authenticity Two definitions of authenticity 1. Real-life performance (RL): the extent to which test performance replicates some specified non- test language performance. Its main concerns are (1) face validity -- appearance or perception of the test and its effect on test performance and test use, (2) predictive utility -- accuracy with which test performance predict future non-test performance. This approach does not distinguish between language ability and the context in which this ability is observed.

Authenticity 2. Functionality or illocutionary purpose or interactional/ability (IA).Ability: theories of verbal abilities. Iinteractional: communicative language use.

Real life language use and authenticity Proficiency and authenticity are effectively synonymous. Direct proficiency test: the testing format and procedure attempts to duplicate as closely as possible the setting and operation of the real-life situations in which the proficiency is normally demonstrated. Indirect test: they may in many cases bear little formal resemblance to linguistic situations that the student would encounter in real life.

Real life language use and authenticity Because of the difficulty of replicating non- test performance in tests, authenticity is described as a continuum between direct and indirect. Validity: The primary validities of RL approach are predictive utility, face validity and content relevance and coverage.

Criticisms of the RL approach Failure to distinguish ability from behavior: competence can not be tested in any direct way. We can measure it through manifestations of it in performance. Inadequate basis for validation: Face validity is the mere appearance of validity and that face validity judgments are naïve because appearances in testing are treacherous, and well-established deceivers.

Authenticity as interactive language use If we could develop a means of classifying test tasks on the basis of dimensions, or factors that we abstract from authentic language use, we should be able to characterize the relative authenticity of a given test task in terms of its potential for generating an authentic interaction with the abilities of a given groups of test takers.

Case study: RL and IA in oral proficiency testing The RL and IA have distinct implications for both the design and interpretation of a given type of language test. RL: ILR Language Skill Level Descriptions ACTFL Proficiency Guidelines OPI (Oral Proficiency Interview) IA: Oral Interview Test of Communicative Proficiency in English

ILR Level Two Can handle with confidence but not with facility most social situations including introductions and casual conversations about current events, as well as work, family, and autobiographical information; can handle limited work requirements, needing help in handling complications or difficulties. Can give directions from one place to another.

ACTFL Advanced Level Able to satisfy the requirements of everyday situations and routine school and work requirements. Can handle with confidence but not with facility complicated tasks and social situations, such as elaborating, complaining, and apologizing. Can narrate and describe with some details, linking sentences together smoothly. Can communicate facts and talk casually about topics of current public and personal interest, using general vocabulary.

IA: Grammatical competence pp

Difference between RL and AI Different views of language proficiency. RL: language proficiency is viewed as a unitary ability. AI: three main scales—grammatical competence, pragmatic competence and sociolinguistic competence, corresponding to the different components of language ability.

Difference between RL and AI Relationship between the contextual features of the elicitation procedure and the ability to be measures. RL: there is no distinction between the ability to be measured and the features of the context in which language performance takes place. AI: scales are defined in terms of levels on the various component abilities.