Presentation on theme: "UKFIET International Conference on Education and Development 15-17 September, 2009 Oxford, England The Politics of Formal Examinations: language policy."— Presentation transcript:
UKFIET International Conference on Education and Development 15-17 September, 2009 Oxford, England The Politics of Formal Examinations: language policy and progress Convenors Pauline Rea-Dickins Guoxing Yu
2 The symposium Rationale: all researching in contexts where: –English as an L2/L3 is used in high stakes formal examining contexts: SSA, Malaysia, USA & UK –Politics & policy impacts on issues of fair & ethical assessment –Policy is intended to lead to progress & equitable distribution of resources & opportunities for ALL learners –Progress?
The symposium Dr Ong Saw Lan (Universiti Sains Malaysia ) Narrowing the science achievement gap with the dual-language test-booklet Professor Jamal Abedi (University of California, Davis) Impact of language factors on the assessment outcomes of English language learners
4 UKFIET International Conference on Education and Development 15-17 September, 2009 Oxford, England The Politics of National Examining Processes: the case of languages in Zanzibar Abdulla H. Mohamed, Pauline Rea-Dickins, Guoxing Yu Student Performance in National Examinations: the dynamics of language in school achievement (SPINE) www.bristol.ac.uk/spinewww.bristol.ac.uk/spine (ESRC/DfID RES-167-25-0263)
5 Introduction Empirical data & documentary evidence reflecting some of the realities of national examining processes: –Student performance on exam items –Statistical data on student exam performance –Implementation of classroom assessment The policy & the politics: historical perspectives: focus on CWA Conclusions : policy & politics & issues of justice in examining –progress? 5
6 Student performance on exam items English RC Qn: How whales resemble man 45 students took this item: 35.6% = no answer 26.7% = wrong answer 28.9% = partially correct answer 8.8% = correct answer
7 Interview: D1 who didnt answer Q3 explains D1:because I did not understand by this this … resemble (lines 115-117) Int:If I tell you that resemble means to look like … can you do the question now? D1:Yes Int:OK so whats the answer? D1:Man … is warm blooded … and whales also … whales have lungs and man also have lungs … (122-133)
8 Biology: responses to original item No answer = 67.4% Wrong answer= 21.7% Partially correct answer = 6.5% Correct answer = 4.4%
11 Biology: on locusts Questions: a) In which picture do you think the locust will/may die? b) Why do you think it will/may die?
12 Original item modified: Greater contextualisation Simplification of instruction 2 structured parts: A & B Visual clues to support information retrieval Rephrasing of the item Altering item layout
13 Results Original item –Only 32.6% of students wrote an answer –Just under 11% gave a partially correct or correct answer Modified item –100% responded to this item –42% gave a partially correct answer to Part A –53% gave a partially correct answer to Part B
14 Changes in student response OR Sc Written responses on Modified Item MOD Score Comment (Neil Ingram: biologist) H30In picture A the locust may/will die I think it will/may dies because the locust get its breathing by using its body 2 + 2Understands that locusts breathe using the body C21I think it will/may die because the boy is dipping the locust in the water to all the bodies with its trachea that used to respiration as a respiratory surface of a locust 2 + 2Getting very close to a very complex answer I think a very able pupil indeed. The right answer but with great difficulty in expressing this in English
Means of Form II Exam Results (MoEVT) 15 KISW Islamic studies BIOCHEMMATHPHYS 20044937.118.430.821.125.8 20054644.322.634.215.325.4 20063647.921.534.315.525.1 200735.544.521.532.014.323.9 200850.647.524.534.816.429.1
16 ENG and MATH Total: 64.717, school=29.18%, pupil=70.82% ENGLISH explains (90.4- 64.717)/90.4=28.41% of the maths total variance Total=90.4, School=30.75%, Pupil=69.25%
17 ENG and BIO 17 Total=65.646, school=18.17%, pupil=81.83% ENGLISH alone explains (114.968-65.646)/114.968=42.90% of the total variance in BIOLOGY Total=114.968, school=18.79%, pupil=81.21%
18 English and CHEM Total=158.281, school=23.06%, pupil=76.94% ENGLISH explains (275.993- 158.281)/275.993=42.65% of the total CHEM variance Total=275.993, school=22.05%, pupil=77.95%
19 How about KISWAHILI & ARABIC? Although other two languages (Kiswahili and Arabic) are also significant predictors of the students performance in maths, biology and chemistry, it is noted that they are less capable of explaining the variances than ENGLISH. KISWAHILI explains (275.993-88.954)/275.993=31.54% of CHEM total variance, (114.968-76.585)/ 114.968=33.39% of BIO total variance, (90.4-75.678)/90.4=16.29% of MATH total variance. ARABIC explains (275.993-203.452)/275.993=26.28% of the CHEM total variance, (114.968-88.569)/114.968=22.96% of the BIO total variance, (90.4-71.885)/90.4=20.48% of the maths total variance.
20 Summary of the multilevel models (a) It is very clear that ENGLISH is a significant and substantial predictor of the students performance in MATH, BIO & CHEM. The school-level variances explained in the cons models as well as in the models including ENGLISH as the single explanatory variable demonstrated a substantial proportion of the variance is attributable to school factors
21 Summary of the multilevel models (b) Not much improvement in terms of fitness of the models (measured by the change of % of school-level variance in the total variance). Therefore, essential to collect further school- and pupil-level data to examine what factors (e.g. English language learning opportunities at home and at school, academic English proficiency) and how much they account for the variances (in the tradition of school effectiveness studies) – our plan for nationwide data collection using pupil and headteacher questionnaires and vocabulary knowledge test
Coursework assessment as part of national examining: implementation issues In Form II, Teacher Assessment (TA) was implemented based on the following assessment procedures: Class work Oral questions Homework Weekly tests Terminal tests 22
Continuous Assessment Format & Guidelines Term: --------- Class: --------- subject--------- Pupils name Month 1 10% Month 2 10% Month 3 10% Month 4 10% Total C/work 40% Exam 60% Grand Total 100% X5638223052
Coursework assessment as part of national examining: implementation issues The first three activities make up 40% and the fourth makes 60% which when combined make 100% The final score (out of 100% ) obtained is sent to the Department of Curriculum and Examination for further processing (CWA is 25% of overall exam score)
Coursework assessment as part of national examining: implementation issues However: significant variability among teachers: the types of specific activities that constitute teacher assessment the number of activities from which they take marks for each month decisions on how to handle student absenteeism how to get 10% of marks for each month; some picked the one that students did better on, some picked randomly, and some put the average of all marks
Coursework assessment as part of national examining: implementation issues Understanding of what assessment is to be done Awareness of guidelines Percentage of marks that go to the final examination Sharing what teachers know and practice about assessment guidelines among teachers within and across schools Level of monitoring of Teacher Assessment by head teachers.
Continuous Assessment Format & Guidelines Note: Coursework assessment will be obtained from class works, home works and weekly tests or as it will be directed otherwise by the Department or the Ministry. All the marks scored by the pupil from the mentioned activities within a particular month, will be combined and transformed into percentage of ten (10%) before filled in the relevant space in this form. Source: Ministry of Education Working Guidelines: Guideline No. 9 (1995)
Continuous Assessment format & Guidelines Additional notes from another version If the term is less or more than four (4) months, four activities chosen by the teacher him or herself should be picked at equal intervals. The teacher is advised to give as many tests as he can in a term but should choose only four for recording. In getting pupils marks for terminal exam for each subject, the exam itself should contribute 60% and classroom assessments (coursework assessments) should contribute 40%
Coursework assessment as part of national examining: implementation issues The language to be used is as indicated in the lesson plan. These notes appear to contradict with the first version which might be leading to confusion. This version requires the teacher to find average of marks in a month while the second requires the teacher to choose one activity from which the marks can be recorded. Source: Ministry of Education Working Guidelines: Guideline No. 9 (1995)
The Policy & the Politics: historical perpsectives The need to make Assessment procedures more friendly, less competitive and practically focused The prescription that student assessment must: –Include works that can enable students to function effectively in their environments –Cover both student performance in theoretical works and in practical productive works –Be done throughout the year
The Policy & the Politics: historical perpsectives The National Examination Council of Tanzania (NECTA), formalised the new examination system by including projects, character assessment, exercises and tests in the assessment system Teachers were then given responsibility to: measure students ability to use knowledge and skills acquired, in their local environments and, keep ongoing records of their progress
The Policy & the Politics: historical perpsectives Tanzania and Zanzibar Education Policies Both The Tanzania Education and Training Policy (1990:79) & The Zanzibar Education Policy (2006:28) emphasised that continuous assessment combined with the final exam paper would be the basis for awarding of certificates at Secondary Education levels For selection purposes and quality control, consistency between teacher assessment and that of the National examinations was emphasized in The Zanzibar Education Master Plan of 1996-2006
Language Policy and Assessment Students assessment is affected by Language policy because students have to demonstrate knowledge and ability through a language accepted by the system. The Zanzibar Education Policy of 1995, Kiswahili version Section 5.72 (28), The Zanzibar Education Master Plan of 1996 (43), the Education Policy of 2006, all emphasized the use of English Language as a language of instruction for all subjects in secondary education except for Kiswahili and Islamic studies.
Language Policy and Assessment So, English, as a language of instruction in teaching science, maths and so forth at secondary level must be a language of assessment Does this have to be the case?
Policy & Politics: progress for learners & learning? Given that English is politically& policy wise the language of the examinations: No test taker shall be harmed by the test (Hamp- Lyons 1989:13) No learner should be disadvantaged, in the interests of justice for ALL learners Ethical issues re. fairness: need to give individual a fair chance 35
36 Policy & Politics: What progress towards social justice? Impact/Potential Disadvantage (examples) Consequences/Injustice: (examples) Learners do not engage or respond poorly in examinations Subject area (e.g. Biology, maths) construct can only be assessed where a linguistic construct has been successfully negotiated Loss of self-esteem & motivation for learning Learners fail to reach their potential (glass ceiling effect) or fail altogether Leave school as unsuccessful (e.g. at end of Basic Education) Unequal access to available resources, educational experiences & work opportunities Unskilled & unable to join the workforce in turn contributing to social & economic deprivation
37 Policy & Politics: What progress towards social justice? Impact/Potential Disadvantage Consequences/Injustice Teachers do not use full range of LTA procedures & processes Use of inappropriately constructed assessment frameworks Inaccurate CWA of learners CWA implemented as a series of tests Learners not fully supported in their language & content knowledge development & fail to reach potential Test performance valued over learning Leave school with poor educational outcomes Inadequate d-base for decision making about student & learning progression