Presentation is loading. Please wait.

Presentation is loading. Please wait.

American Association for Applied Linguistics Conference 6 th -9 th March, 2010 Sheraton Atlanta Hotel, Atlanta, Georgia Investigating the Dynamic Interplay.

Similar presentations


Presentation on theme: "American Association for Applied Linguistics Conference 6 th -9 th March, 2010 Sheraton Atlanta Hotel, Atlanta, Georgia Investigating the Dynamic Interplay."— Presentation transcript:

1 American Association for Applied Linguistics Conference 6 th -9 th March, 2010 Sheraton Atlanta Hotel, Atlanta, Georgia Investigating the Dynamic Interplay between Languages in the Fair Assessment of School Subject Knowledge Professor Pauline Rea-Dickins and Zuleikha Khamis

2 2 What is SPINE? Student Performance in National Examinations: the dynamics of language in school achievement (SPINE) www.bristol.ac.uk/spine (ESRC/DfID RES- 167-25-0263)www.bristol.ac.uk/spine Bristol team: Pauline Rea-Dickins, Guoxing Yu, Oksana Afitska, Rosamund Sutherland, Federica Olivero, Sibel Erduran, Neil Ingram, Harvey Goldstein Zanzibar team: Zuleikha Khamis, Abdulla Mohammed, Amour Khamis, Mohammed Abeid, Shumba Said, Haji Mwevura

3 Overview of Presentation Background & issues Recent research & initiatives SPINE empirical studies & findings Some conclusions 3

4 Background Language use in schools Kiswahili = MoI in primary English = MoI in secondary Kiswahili, English & Arabic taught as subjects a primary & secondary The reality of the classroom is that both Kiswahili and English are used Evidence of policy changes English as MoI to be brought in earlier at primary (std4) but commitment to raising children bilingually 4

5 Background Examining Formal high stakes examinations at end of 2 nd year of secondary in English Levels of achievement at end of Basic Education alarmingly low > 50% of school aged children leave school at the end of Basic Education as unsuccessful 5

6 Background & Issues L2 language academic development Working bilingually in the classroom Monolingual (L1) examining Negotiating double constructs in examinations Monolingual & NS-orientation 6

7 Recent Research Initiatives & Policy Decisions Accommodations for ELL in US (e.g. Work of Abedi; Bailey & Butler; Rivera et al) Teaching & learning of Science and Maths through English: reversal of policy in Malaysia (see also Lan, 2010) Sri Lanka: at O and A level – students are using 2 languages to study & not compulsory to sit the exam in EL2: they can choose (Punch, 2009) Research focus in sub-Saharan Africa more on classroom interactions & subject learning with less attention for the formal examining of this subject knowledge 7

8 Design of SPINE Studies 1.Original examination items Biology, English, Chemistry, Maths followed by interviews with test takers (N=45) 2.Modified examination items Biology, Maths, Chemistry: Greater contextualisation Simplification of instruction Restructuring of questions, e.g. 2 structured parts Visual clues to support information retrieval Rephrasing of the item Altered item layout 8

9 Design of SPINE Studies 3.Developed new examination papers: BI, CH, MT Larger sample of students (N=800+) 3 versions: English only Kiswahili only Bilingual: English & Kiswahili 4.Process Studies 5.Vocabulary test: measure of language proficiency 6.Student questionnaire 7.National examination longitudinal data 9

10 Original Items: Summary of Findings Very low mean scores across subjects Student achievement lowest in Maths Significant differences in achievement across the subjects Strong correlation between English (exam + vocabulary test) & performance in other subjects Factors affecting performance from learner interviews: Not understanding the task, specific words & phrases or the meaning of tables and diagrams Partial knowledge of topic area & question type Low levels of students language proficiency 10

11 Orginal Items 11

12 Modified Items – smaller sample Original question was interpreted in 3 different ways with learners including: The 14 year olds (3+2+5+4+2=16) Cells on the left containing 14 (10, 11, 12, 13) Cell below the4 cell containing 14 (which is 2) Modifications included: Instruction simplified Changed under to younger 12

13 13 Maths: original & modified question

14 Modified Items: some findings Only 1 student made a mistake in interpreting the item (below/younger) All 6 who got part (a) correct provided correct translation 3 our of 4 who got this wrong did not provide translation 1 who gave correct translation got part (a) wrong 14

15 Modified Items: some findings Increase in student response rates Increase in accuracy of responses Correspondence between ability to provide a correct translation of task & ability to solve/provide partially correct answer Learners who had difficulty translating task generally performed poorly Evidence of linguistic factor + other factors 15

16 16 LanguageBiologyChemistryMaths Kiswahili184183184 Bilingual: K + E152 English171 Total507506507 New Examinations: 3 Versions – random allocation

17 17 Qualitative Analysis of Student Responses Qn 5: Write about an animal you have studied Name of the animal: What does the animal look like? Where does the animal live? Describe how the animal eats?

18 Lower Scoring Group: answers in ENGLISH version Direct copying of English words + incomprehensible answers Example: the animal name the kingdom (1); the animals look like because (189); does the animal live caw, goats, dog; the describe the animal eats, caw, goats, camel (9) Incomplete sentences were given as answers but learners failed to show the actual subject knowledge or the ability to express it in English Example: the animal live in the ______(1)

19 Lower Scoring Group: answers in ENGLISH version Repetition of the identical chunks of language (e.g. names of the animals) together with bare copying of parts of the question showed learners partial understanding of the subject/question content (he/she understood that it had something to do with animals but WHAT exactly about animals he/she could not express possibly due to language/subject knowledge limitation) Example: lion dog caw goats crocodile; bat snail; does the animal live caw, goats, dog; the describe the animal eats, caw, goats, camel (9); 19

20 Lower Scoring Group: answers in ENGLISH version Vague responses, e.g. included more information than was required - often unnecessary or irrelevant failed to articulate clearly what they meant to say Examples: the animal eats: –man and sheep eat by mouth –plant used to gaseous exchange No response at all: language or subject knowledge limitation? 20

21 Higher Scoring Group: answers in ENGLISH version Correct answers in more than one part of the question also showed ability to provide: (a) complete simple sentences in L2 (even though often not entirely grammatically accurate) which represented fairly meaningful responses Example: Animal live in bush (13) or (b) complete compound (extended, longer) sentences in L2 which also represent ed fairly meaningful responses Example: Animal live in the forest or in the water (33) 21

22 Lower scoring group - answers in KISWAHILI version Ability to construct complete sentences making sense in their own right Example: It eats through the human body (33) It lives on land (383) Compound sentences of variable length Example: it lives in water but we cannot see it without using microscope (195) Ability to contrast Example: The animal is round it is not straight (377) 22

23 Lower scoring group - answers in KISWAHILI version Construct unfolds the sense learners want to make Example: duponi for tumboni i.e. stomach (377), bafu for pafu i.e. lung (395), mstu for msitu i.e forest (14) Kiswahili spelling substitutions Example: sell for cell (383), enimal for animal (33) Adopting rhythmic similarity in recalling the required vocabulary Example: gras mamalia to mean class mammalia 23

24 Lower scoring group - answers in KISWAHILI version Making comparative sentences Example: The animal is as small as a lion and it is of average size and has different characteristics (14) Use of phrasal structures which still make sense Example: kwenye mwili wa binadamu = in the human body (33) Overall: no occasion was observed where a student has left a gap Variable chunks of words and utterances; elaborations Example: It eats the food that we put in our stomachs. It digests the food (377) 24

25 25 Mean scores for each subject across the different versions:

26 Comparison of student performance across the 3 different versions and per subject 26

27 27

28 28 Schools A-F – limited exposure to English Tests of significance: comparison of students performance across the 3 different versions and per subject

29 29

30 30 Schools G-H – likelihood of increased exposure to English Tests of significance: comparison of students performance across the 3 different versions and per subject

31 31

32 32 Schools G-H – increased exposure to English

33 33 Findings from National Form II Data: summary of the multilevel modelling analysis It is very clear that ENGLISH is a significant and substantial predictor of the students performance in MATH, BIO & CHEM. But: the school-level variances explained in the cons models as well as in the models including ENGLISH as the single explanatory variable demonstrated that a substantial proportion of the variance is attributable to school factors

34 Some Conclusions & Challenges Most affected lower achieving learners What evidence is there of the impact of research findings such these on actual testing and assessment policies and practices? We must do something to overthrow the dominant & unquestioned role that EL1 has in many examining contexts: time for a coup! Time to reconceptualise constructs: NS orientations Try out supportive & context sensitive approaches to assessment 34

35 35 Enhancing Learning & Social Justice Impact/Potential Disadvantage (examples) Consequences/Injustice: (examples) Learners do not engage or respond poorly in examinations Subject area (e.g. Biology, maths) construct can only be assessed where a linguistic construct has been successfully negotiated Loss of self-esteem & motivation for learning Learners fail to reach their potential (glass ceiling effect) or fail altogether Leave school as unsuccessful (e.g. at end of Basic Education) Unequal access to available resources, educational experiences & work opportunities Unskilled & unable to join the workforce in turn contributing to social & economic deprivation


Download ppt "American Association for Applied Linguistics Conference 6 th -9 th March, 2010 Sheraton Atlanta Hotel, Atlanta, Georgia Investigating the Dynamic Interplay."

Similar presentations


Ads by Google