1. © 2005 Assessing the language proficiency of young learners in the Dutch classroom: developing and implementing tasked-based language tests Marleen Colpin & Machteld Verhelst

2. © 2005 CONTENTS PART I (Machteld Verhelst) Background & principles Assessing very young children’s language proficiency PART II (Marleen Colpin) Developing and implementing tasked-based language tests in the classroom Conclusion

3. © 2005 Its task: promoting equal educational chances for all learners Most important target group: learners at risk in their educational career (immigrants and learners from low social classes) Providing support to the field of primary, secondary and adult education Centre for Language and Migration

4. © 2005 Focus on successful language performance in order to function in relevant, communicative situations (school, society) Language as a device, not a goal in itself Becoming proficient through carrying out realistic, relevant and motivating tasks Task-based approach for every learner regardless of his/her language background Task-based approach of CTM

5. © 2005 Task-based assessment Task is central concept, not construct of LP Not to elicit components of system Interest in performance on task itself

6. © 2005 Principles Three principles concerning development of language tests and assessment instruments: 1.Embedded in education 2. Implementation 3. Impact

7. © 2005 Three interrelated basic questions to start with: WHY is a test needed?  purpose WHAT should be tested?  content HOW should language proficiency be tested?  format

8. © 2005 To monitor the language development process and diagnose possible problems To provide feedback and specific support on the basis of diagnostic information  perspective of learner: to avoid problems and/or to overtake these  perspective of teacher: to adjust own education To take decisions To inform parents... WHY?

9. © 2005 Why do teachers evaluate?

10. © 2005 WHAT? Language performance in relation to Intermediate goals / Governmental attainment goals / Framework of Reference for young learners No global level system such as the CEFR but goals in terms of the language tasks that learners should be able to perform at a certain age within and outside the school context

11. © 2005 Which goals do teachers assess?

12. © 2005 HOW? The most appropriate procedures/instruments to test/assess with regard to - WHY you want to test - WHAT you want to test - but also the GROUP of PUPILS that will be tested principles of good testing and assessment: validity, reliability, efficiency and practicality

13. © 2005 Assessing very young learners’ language proficiency: a broad view… Social Cognitive Language

14. © 2005 WHY? Purpose: to detect children at risk in order to create a more powerful language environment

15. © 2005 WHAT? ‘Attainment’ or ‘developmental’ goals  Language use  In the school domain  Comprehension  No correctness

16. © 2005 Examples of attainment goals end kindergarten Listening A child can understand an oral instruction about a concrete physical action in a here- and-now situation, and is able to show its comprehension by reacting adequately. Speaking A child can answer questions about its own feelings, intentions and interests, in a communicatively adequate way.

17. © 2005 What do teachers test?

18. © 2005 HOW? Specific demands concerning: (1) Validity, reliability and bias Tasks Test format Cultural/social bias  Avoiding typical culture-bound elements  Themes and contexts that all children know

19. © 2005 (2) Efficiency and practicality Tests demand much of the concentration of young children  as short and as natural as possible  variation and motivation

20. © 2005 (3) Embedding in educational context Possibility to draw educational consequences? Education effects visible? Evolution visible? Enough differentiation?

21. © 2005 Example 1: KOBI-TV Age 4 Language proficiency in the school domain Integrated oral skills 27 tasks: real language use situations Administration within the natural class environment of the children Allows monitoring during the school year

22. © 2005 1 0 0

23. TASK 7

24. © 2005 ‘Take three green blocks. Put these blocks in a row.’ TASK 17

25. © 2005 TASK 24

26. © 2005 TASK 27

27. © 2005 The standards of KOBI-TV DecemberMarchJune 111619 2325 DecemberMarchJune 182224 Immigrant children Flemish children All children

28. © 2005 Developing and implementing tasked-based language tests in the classroom Marleen Colpin

29. © 2005 VLOT Volgsysteem Lager onderwijs Taalvaardigheid  to monitor the language development of primary school pupils (8-12 year olds, L1 and L2 learners)  a battery of 60 tests  designed for teachers  in relation to the official attainment goals

30. © 2005 Examples of attainment goals for the end of primary education Listening The pupils can understand and structure the information from an explanation or an instruction of the teacher. Writing The pupils can write a letter to a familiar addressee to communicate a personal message or experience.

31. © 2005 Relevant and authentic assessment tasks which relate to the official attainment goals Criteria/parameters for difficulty: Text processing level Text type Public Familiarity of topic and theme Vocabulary Reading difficulty level (AVI-level) VLOT

32. © 2005  Unequal division between receptive and productive tests  Tasks with a communicative goal  Confronting children with language they encounter in real life (their world): motivation 60 tests:  1 test for each grade (5 grades)  for the 4 main language skills  at 3 moments during the school year VLOT

33. © 2005  Direct testing to enhance content validity  Practical and efficient to be administered  Target group: L1 and L2 learners  avoiding bias  Comparable test results to monitor pupils  Need for efficient rating scheme that guarantees objective and reliable rating VLOT

34. © 2005 Example test: Birthday party Oral instructions from the test administrator/ teacher: “… You will now call X (name) on the phone to ask him/her whether he/she can come. You will tell him/her everything he/she needs to know to come to your party. So you will have to mention when and where the party will be, what time it begins and ends, if the children need to bring something, e.g. a present, …. I will play X. I will first give you a little time to plan, so that you can prepare what you are going to say.” VLOT

35. © 2005 Comparable test results Results gathered from a group of more than 600 pupils in 32 schools representative for Flemish population  5 standard categories A = good B = average good C = average weak D = weak E = very weak VLOT

36. © 2005 VLOT

37. © 2005 Rating table Birthday Party task 1.Why? (the friend is invited)1/0 1/0 2.Where? (the party takes place) 1/0 1/0 3.When? (the party takes place) 1/0 1/0 4.At what time? (the party starts) 1/0 1/0 5.What to bring along? 1/0 1/0 + speaking fluency: 1/0 contentform VLOT

38. © 2005 Implementing task-based tests Multidirectional relationships between tests, educational programme and learning goals ><tradition of focus-on-formS language teaching methodology ><control +high impact and washback on views and practice of education

39. © 2005 TASAN Taalvaardigheidstoets Aanvang Secundair Onderwijs Anderstalige Nieuwkomers  Test entrance secondary education non- native newcomers  Purpose: to yield information about the linguistic skills of a non native newcomer after one year preparatory class and to diagnose problematic aspects

40. © 2005 Parameters of task complexity  Parameters concerning the world represented in the task (e.g. ‘here-and-now’ versus ‘there-and-then’; amount of visual and linguistic support)  parameters with regard to the tasks’ cognitive processing demands  parameters with regard to linguistic complexity (vocabulary, syntactic structures, length of text) TASAN

41. © 2005 Task Complexity Indicator TASAN

42. © 2005 TASAN In-service trainings  Links between tasks in the test and attainment goals  Tasks in the test = examples of the tasks to be used in language education  Illustration of manipulation of complexity

43. © 2005 Teachers developing tests? Difficulties: - finding or selecting proper source materials - avoiding test bias - establishing the difficulty of the test - constructing independent items - constructing objective and task-relevant checklists for scoring productive skills Arguments pro: +Carefully consider attainment goals +Elaborate professional proficiency with regard to testing

44. © 2005 Broad concept of evaluation  on a permanent base as well as at specific moments throughout the school year  on products the pupils produce as well as on their acquisition/learning process  with more traditional tests as well as more ‘alternative’ evaluation instruments, such as observation instruments, portfolio,...  by the teacher as well as the pupils themselves or other pupils

45. © 2005 Teachers’ assessment skills Insight in what relevant tasks are and what constitutes their complexity  more refined observation skills  base assessment of pupils’ language proficiency upon a rich and varied base of ‘real’ language use situations in the classroom

46. © 2005 Conclusion Give teachers insight in: What attainment goals are claimed to be evaluated? How was the test constructed? What aspects of learner performance are assessed?  Task-based assessment as a powerful tool to promote more functional, performance-based assessment, and task-based language education