The Development of the E 8 Listening Test Claudia Mewald Dave Allan Andrea Kulmhofer-Bommer
Construct: WHAT E8 standards for listening descriptors based on CEFR: (A1) A2 - B1 aligned with Austrian national curriculum topics listed in curriculum text types What the test is based on
Construct: HOW TOP-DOWN meaning making from input texts interactional transactional interactional focus on message focus on listener BOTTOM-UP What the test focuses on
Specifications & formats Baseline study 2009 Section Focus Text-type (input) Item format Length input text (words) No of texts No of items 1 Direct meaning comprehension or Inferred meaning comprehension descriptive, narrative, expository, argumentative, instructive, literary Three domains: personal, public and educational 4-option MC max. 100 per text 10 2 200-500 (max. 500 for both) 1 - 2 played twice 11-12 20
Length input text (words) Adaptations: 2013 E8 Test Section Focus Text-type (input) Item format Length input text (words) No of texts No of items 1 - 6 Prediction Inferring meaning Relating utterances to contexts Identifying speaker attitude Listening for gist/main idea Listening for specific information/detail(s) descriptive, narrative, expository, argumentative, instructive, literary Three domains: personal, public and educational 3-option MC matching gap- filling prediction 70-100 36-48 34-46 played once
Length input text (sec.) Adaptations: 2019 E8 Test Section Focus Text-type (input) Item format Length input text (sec.) No of texts No of items 1 - 3 Overall listening comprehension: listening for gist/main idea Implied meaning comprehension: inferring meaning from context and identifying speaker attitude Detailed listening comprehension: Listening for important information and supporting/specific details Advertisements, announcements, messages, instructions, directions, narratives, radio news reports, podcasts, interviews, radio plays, discussions Three domains: personal, public and educational 3-option MC, matching (5/6), T/F, testlets approx. 30-60 sec per input text 27-30 44-49 played once
The Development of the E 8 Speaking Test Claudia Mewald
E8 SPEAKING TEST COMMUNICATIVE FUNCTIONS ORAL PRODUCTION PRAGMATIC A2 real-life process co-constructed knowledge & ability to communi-cate INTERVIEW MONOLGUE A2 C O N T E X C E F R S O C I L N G U T T E X Y P S C U R I L UM D E S I G N A2 B1 SHORT DIALOGUE (2013) LONG DIALOGUE communi-cative compe-tence knowledge compe-tence use TEST LINGUISTIC SPOKEN INTERACTION TOPIC AREAS
Interlocutor training Assessor training Interlocutor training S C E N A R I O Assessment scale Test specifications Prompt development CEFR linking
Washback of standardised testing in the subject English at lower secondary level Claudia Mewald
Level 1 Level 2 Level 3 - past supporting measures Organising Institutions goals – educational change teacher education and development perceived needs of teachers Level 2 Executing Institutions supporting measures - past - current - future Level 3 Implementing Institutions impact on teaching impact on assessment impact on learning
Washback hypotheses by Alderson & Wall (1993) 15 Washback Hypotheses 326 teachers, 46 head teachers, 140 pupils, 48 parents 26 schools, 58 teachers, 95 lessons observations level 3 Washback hypotheses by Alderson & Wall (1993) 15 Washback Hypotheses Alderson & Wall (1993) interviews levels 1-3 questionnaires level 3 published materials & research, tests, handouts from lessons, lesson plans & long term plans ... 12 experts, 10 head teachers, 27 teachers document research
pupils are not aware of standards specific, strong and positive intended and strong with trained raters some pupils liked the test pupils would like to receive more immediate feedback pupils are not aware of standards they do not benefit from the feedback they lose interest in results writing prompts graded tasks reading and listening strategies speaking writing as gap-filling reading aloud reading or listening strategies teaching testing motivation learning specific but weak positive but short and weak weak