Presentation is loading. Please wait.

Presentation is loading. Please wait.

L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2006 1 Assessing Reading Literacy in the Language of Instruction John H.A.L. de Jong Intergovernmental.

Similar presentations


Presentation on theme: "L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2006 1 Assessing Reading Literacy in the Language of Instruction John H.A.L. de Jong Intergovernmental."— Presentation transcript:

1 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Assessing Reading Literacy in the Language of Instruction John H.A.L. de Jong Intergovernmental Conference: Languages of Schooling towards a Framework for Europe Strasbourg, October 2006

2 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Overview Background of the PISA studies Design of the PISA studies PISA approach to Reading Literacy Some central findings of PISA 2000 Extensions for PISA 2009 Relationships between PISA & CEF Applying CEF in Upper Secondary Vocational

3 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Background of PISA study OECD: Organisation for Economic Co-operation and Development 30 member countries committed to democracy and market economy Provide comparative data and analyses For countries –to compare policy experience; –seek answers to common problems; –identify good practice; –co-ordinate policies. To support economic growth, boost employment, raise living standards.

4 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong History OECD 1948 Organisation for European Economic Co-operation (OEEC), set up with support from the United States and Canada to co-ordinate the Marshall Plan for the reconstruction of Europe after World War II (www.oecd.org/history) Created as an economic counterpart to NATO, the OECD took over from the OEEC Early recognition of importance of education of its Citizens 1985  Gather indicators on investment in education Education at a glance yearly publication 2000 first PISA

5 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong PISA Goals After studying resources & investments in education for a number of consecutive years:  Wish to know the yield of education: at the end of compulsory education how well are students prepared to continue further education or to start working life?

6 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Reading Maths Science PISA assessment schedule

7 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Reading Maths Science PISA assessment schedule

8 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Participation in OECD/PISA PISA YearOECDPartner

9 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Definition Reading literacy is understanding, using and reflecting on written texts, in order to achieve one’s goals, to develop one’s knowledge and potential and to participate in society.

10 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Retrieving information Interpreting text Reflection & Evaluation

11 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Let’s look at Reading In the activity of reading we can distinguish four main elements: 1 there is a reader 2 there is a text (the author is part of the text, but not present in the activity) Both of these are observable. 3 the reader has a goal (a self-chosen reason for reading the text) 4 the reading activity will have a result (a change in the reader: more knowledge, satisfaction, etc ) These are not observable.

12 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Text - Reader - Goal - Result Assessing Reading: - Response- Test taker- TaskText Reading in ‘real’ life - Scoring rules - Score Not chosen, but imposed by test Must be observable Conform to the task Reflects difference between task and observed response End point of process

13 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong These elements should be relevant to the intended trait These elements should interfere the least possible refers to (part of) a text sets task pertaining to the text, requiring particular subskill(s) specifies format for overt response demonstrating application of subskill specifies rules for scoring response A Reading item may require to link information from the text to prior knowledge and experience Only then are score differences likely to reflect differences in true ability on the intended trait: i.e. reliable and valid

14 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Difficulty of comprehension depends on text characteristics Reader Aspects of item difficulty Text Comprehension is the interaction between text & reader ▪ Familiarity ▪ # of elements ▪ Ling.Characteristics ▪ Complexit y Difficulty of task depends on prior knowledge & experience of candidate – Personal / Private – Formal / Public + + Evaluate / Hypothesize Find / Connect /Assess Comprehension Difficulty of item depends on scoring rules Difficulty of item depends on skills required to perform task Scoring Guide Specificity Difficulty of item depends on response format Response Format

15 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong

16 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong

17 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong PISA 2000: Relation SES & Reading SESREADING High540 Medium500 Low460

18 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong

19 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Mean PISA 2003 scores per educational track in the Netherlands

20 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Mean Reading Literacy score for Dutch-born students and for immigrant students

21 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Language proficiency in upper secondary vocational education CINOP Study using CEF what is the actual language proficiency of students? is there a discrepancy between actual and required language proficiency for further education and professional occupation?

22 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Research design -written questionnaires for language teachers and vocational teachers, N = 210 -self assessment on language proficiency for students, N= 345 -followed by a test for reading, N= 328 -oral interviews with teachers, staff members and management, N= 40

23 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Teachers’ views 60 to 70 % estimate language proficiency (reading A2-B1) insufficient for school 10 % estimate the language proficiency on a higher level than needed for school 80 % estimate the language proficiency insufficient for professional occupation

24 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Self assessment by students > 2/3 of students estimate their reading proficiency level at B2 In reality:  7 % at A1  24 % at A2  52 % at A2-B1  17 % at B1-B2

25 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong CEF appropriate for description of language of instruction Coherent and workable structure for teachers (contributes to improvement of their professional competences) Competence based descriptions more suitable for language of instruction than linguistic descriptions of L1 Descriptors applicable More transparency and motivating for students (specially in combination with a language portfolio) Offers opportunity for reflection on language learning by students

26 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong Shortcomings of CEF in describing language of instruction in vocational education Can do statements are geared at academic learning routes: not well adjusted to describing specific tasks in vocational education and specific tasks in the workplace More situations and examples have need to be offered for a range of vocational sectors

27 L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong “Milestones” st PISA Cycle: Reading Major Domain 2002 Publication Reading for Change 2004 CINOP Dutch in Vocational Education 2006 ALARM: Language skills at risk


Download ppt "L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2006 1 Assessing Reading Literacy in the Language of Instruction John H.A.L. de Jong Intergovernmental."

Similar presentations


Ads by Google