1. Language Assessment 4 Listening Comprehension Testing Language Assessment Lecture 4 Listening Comprehension Testing Instructor Tung-hsien He, Ph.D. 何東憲老師 the@tea.ntptc.edu.tw

2.  Auditory Skill vs. Oral Skill a. Closely Linked in Normal Speech Situations a. Closely Linked in Normal Speech Situations b. Listening Abilities May Be Developed Beyond Range of Speaking Or Writing Therefore, listening comprehension testing is needed.  Purposes of Testing Listening Comprehension a. To test the ability to distinguish phonemes

3. b. To test listening comprehension of utterances (it could be a single sentence, a paragraph, or even a lecture).  Threats to Validity & Reliability of Listening Comprehension Testing: a. Abilities to discriminate phonemes do not imply abilities to understand utterances; b. Testing materials may not simulate real-world speeches in terms of lack of contextual clues and/or spoken (redundant) features like

4. hesitations, time filler (like Well, N…, Kind of), false starts ; c. Administrations of testing may not include communication strategies used such as asking again or body language; d. Certain types of items may test intelligence or memory in addition to listening comprehension proficiency (Think about “lecture” type of testing).

5. e. If the distractors or any other part of testing items are written, these items also test other skills (reading comprehension, vocabulary, lexicon,... ) in addition to listening comprehension  Types of Items: 1. Phoneme (Sound) Discrimination Tests Type 1: a. Word sets in isolation (p. 33): (1) Minimal Pairs

6. (2) Minimal pairs with an extra word: Two Identical Sentences & One Different Sentence (with only one different word) (p. 66) Type 2: Words in context (p. 34) (1) A heard sentence with two pictures (2) A heard sentence with a written sentence with an empty blank (target word)

7. Rules of Thumb for Item Writing: a. Use Recordings (e.g., movies, tapes... ) b. Avoid extremely difficult lexical items b. Use simple structures and lexicon c. Use the “normal” & consistent stress & pitch patterns d. Avoid switching or changing stress & pitch patterns e. Use “Standard American English Pronunciation” f. Simulate spontaneous speeches as possible

8. Criticisms: a. Appropriate for diagnostic purposes (1) contrast of individual phonemes of L2 (2) contrast of individual phonemes between L1 & L2 (3) identification of difficulties b. For Type 2-1 (A heard sentence with two pictures), what else has been tested as well? c. For Type 2-2 (A heard sentence with a written sentence with an empty blank (target word), what else has been tested as well? (Vocabulary & Syntax)

9. 2. Sentences & Dialogues: Type 1: Sentences (p. 37) (a) Questions: A heard question with four distractors used as answers to the question (Hint: Testees have to understand the entire questions) (b) Statements: A heard statement with four complete sentences as distractors (Hint: Testees have to understand the target word in the statement)

10. (c) A heard sentence with four pictures (Hint: Testees have to understand the entire sentence and the implications of the four pictures without reading any passages) Type 2: Dialogues: A heard short speech & Distractors (p.40)

11. a. Short samples of speeches & dialogues b. Appropriate for administrations in language labs c. Resemble natural discourses without redundant features d. Evaluate “hearing” not “interpreting” e. Avoid testing memory

12. Rules of Thumb: Keep the problems in the stems rather than in the alternatives. Criticisms: a. For Type 1, what else has been tested? (Grammatical & Lexicon) So, what will happen? b. For Type 2, what else has been tested? (Ability to select an appropriate reply to a stimulus & all abilities tested in type 1) 3. Testing Listening Comprehension Through Visual Materials:

13. Features: a. Use pictures or photos to replace written passages b. May be appropriate for young testees c. Pictures or photos must be clear enough so that problems can be easily spotted from them d. May also test cognition/intelligence Types of Items: a. Type 1: One Photo First then Questions

14. b. Type 2: A Few Photos First then Questions c. Type 3: A Set of Photos First then Questions d. Type 4: A Few Diagrams First then Questions e. Type 5: Following Instructions f. Type 6: Drawing Rules of Thumb: a. Always give clear instructions & examples b. Define the acceptable drawings c. Avoid testing intelligence

15. Criticisms: a. Be cautious about the testees’ ability to draw b. Hard to score drawings c. May test intelligence 4. Lectures & Talks: Lecture then Questions Rules of Thumb: Rules of Thumb: a. Avoid testing memory b. Provide clear instructions & Examples

16. c. Provide real speeches with normal delivery rate, pauses, and breath segments. 5. TPR

17. a. See the reading booklet for detailed information:

18. Rules of Writing Distractors: 1. Target words may belong to more than one part of speech; so it may have different meanings if used as different part of speech (p. 43, E.g. A) 2. For a question used as the stem, think about what can be misunderstood by testees and these misunderstandings can serve as good distractors: See E.g. B 3. For a question with a subordinate clasue used as the stem, think about what can be misunderstood: See E.g. C & D