1. C HAPTER 9: M ACRO – AND MICRO - EVALUATIONS OF TASK - BASED TEACHING R OD E LLIS Presented by: Riris Marlentera, Agnes Lusi Handaru Prastiwi M. Ainur Rizqi

2. W HAT TBLT IS GOING TO DO ? Engaging students in a series of communicative tasks. TBLT is based on a view of language learning that claims a L2 is best learned through learner’s effort to communicate with it.

3. D EFINING OF TASK Every task has 4 criteria: 1. It should be meaningful (semantic and pragmatic). 2. There should be a gap. 3. Learners should rely on their own resources. 4. Language is a means to achieve outcome not as end.

4. D ISTINCTION BETWEEN TASK AND SITUATIONAL GRAMMAR EXERCISE Situational grammar exercises satisfy the criteria 2 and 3 which means there is a gap and learners should rely on their own learning but it doesn’t satisfy criteria 1 and 4 as the outcome is primarily the use of correct language.

5. T ASKS CAN BE DISTINGUISHED IN A NUMBER OF WAYS 1. Unfocused task: designed to help learners to use language communicatively in general. Focused task: are designed to help learner to use language communicatively by using specific language features. 2.Task can also be input providing or output prompting. 3. Tasks can have closed or open outcomes.

6. D ISTINCTION BETWEEN TASK - BASED AND TASK - SUPPORTED LANGUAGE TEACHING Task-based: requires a syllabus consisting of unfocused tasks, the content of the instructional programme is specified in terms of the tasks to be completed. Task-supported language teaching: utilizes a structural syllabus and typically involves (presentation, practice, production). According to Widdowson: task-supported language teaching is likely to result in encoded usage rather than realization as purposeful use.

7. A PPROACHES TO EVALUATING TBLT Macro-evaluations Micro-evaluations

8. M ACRO EVALUATION Macro-evaluation can be defined as evaluation that seeks to answer one or both of the following question: 1. to what extent was the programme/project effective and efficient in meeting its goals? 2. in what way can the programme/project be improved?

9. D EFINITION OF MACRO EVALUATION an evaluation carried out for accountability or developmental purposes by collecting information relating to various administrative and curricular aspects of the programme including teaching materials.

10. M ICRO EVALUATION A narrow focus on some specific aspects of the curriculum or the administration of the programme. macro-evaluation may eventually emerge bottom- up from repeated micro evaluations.

12. M ACRO - EVALUATION OF TASK - BASED TEACHING

13. Bretta,Davies sought to compare the learning outcome of learners involved in the Project (experimental group) with those in traditional classes (control group). Tests that ‘favoured’ the experimental group (i.e. task-based test). Those that ‘favoured’ the control group (i.e. structure-test) and three neutral tests (i.e.contextualized grammar, dictation, reading/listening comprehension)

14. In the neutral test,the experimental group scored higher than the control group. on the group –biased test the experimental group did better on the task-based tests and control group scored higher in the structural test. they conclude that the results of the evaluation support the claim that task-based instruction produce different results from form- focused instruction. And this is reflected in the task-based learners superior acquisition of the structure that have not explicitly taught and also in their ability to deploy what they have learned more readily.

15. Bretta collected historical narrative from 15 teacher and then rated it: 1) Orientation : lack of understanding task based and failed to implement it. 2) Routine : understanding the rationale and implement it. 3) Renewal: adopt a critical perspective and demonstrate the weakness and strengths. Bretta found that 40 % of the teacher were at level 1, 47 % at level 2 and 13 % at level 3. He finally concluded that what he had done in the project cannot be assimilated by regular teacher and this is because lack of proficiency.

16. There have been a number of evaluation: The study by Li (1998): context: the introduction of communicative language teaching into secondary school in South Korea where traditionally a grammatical syllabus has been used. purpose: to investigate the perception of the teacher of South Korea of implementation of CLT. evaluation method: questionnaire :with Likert- scale items and open question administered to 18 secondary school teacher. Finding: report difficulties in using CLT in Korea: 1) difficulties caused by teacher. 2) Student 3) educational system 4) CLT, lack of appropriate assessment instrument

17. The study by Mitchell and Lee (2003): Context: two settings explored: 1. 11-12 year old English students in secondary school learning French 2. 10 -11 year old students in a Korean primary school learning English Purpose: to examine how the communicative approach was interpreted and how the role of good language learner was constructed. Evaluation method: twenty recorded lessons- interviews and back up documentation. Findings: both teachers reflected a weak interpretation of CLT: 1. They aimed to provide learners with a fixed body of predetermined expressions and little opportunity for creative language. 2. Teacher E adopted an egalitarian, un-differeinitiated approach whereas teacher K privileged the more able students to provide the good model

18. The study by Carless (2004): Context: primary school classrooms in Hongkong. The three teachers in these classroom implement task-based syllabus. Purpose: what are the teachers attitude and how they are attempting to implement task-based syllabus? Evaluation method: 1. classroom observation-field notes and transcriptions 2. six-semi structured interview 3. an attitude scale(data analyzed qualitatively) Findings: the tasks represented language practice activities rather than genuine practice. They identified 3 key issues: 1) use of mother tongue 2) discipline challenges 3) target language production

19. The study by Butler (2005): Context: the governments of Korea, Japan, Taiwan have introduced English teaching with the expectancy that the focus will be on the oral communication. Purpose: to identify teachers concerns regarding communicative activities. Evaluation method: multi vocal ethnography involving presenting teacher with videotaped scenes. Findings: 1) teachers in different countries employed similar activity with different motives which led to different activities 2) teachers expressed concerns about how to develop communicative activities that were suitable for higher grade students 3) class harmonisation seen problematic especially in Japan

20. The study by McDonough and Chaikitmongkol (2007): Context: replace the traditional form focused course with task-based course in Thailand. Kkey focus of course was the use of English for international communication. Purpose: 1. What were Thai teacher and learner reactions toward task-based course? 2. If they had any concerns how were these concerns addressed? Evaluation method: task evaluations-students kept learning notebooks-observations by teacher and participant- course evaluation-interviews-field notes Findings: A. teacher and learner reaction: 1. increased learner independence 2. course content 3. real world relevance B. addressing participants concerns: 1. revision undertaken to help participants adjust to the course 2. providing learner support 3. managing course materials.

21. An inspection of (purpose)column suggests that these evaluation studies had 2 general aims: 1. to identify teachers perception and attitudes to task-based teaching. 2. to examine how the teacher implemented TBLT. Butlers observation support sociocultural view of ‘task’: task : the actual materials that compromise the workplan for the activity. activity: the learners behaviour that ensues when learners perform tasks.

22. Carless reported that teachers he investigated had mixed and confused the notion of ‘task’. McDonough and Chaikitmonkgol have the most to say about materials. One of the problem they identified is how to interlink task-based materials that specifically developed for the university level with the materials from commercial book. Another issue is the quantity of material. They conclude that it’s the curriculum that dictates the use of textbook not the textbook.

23. Finally we conclude that such macro-evaluation shed light on the viability of TBLT. It offers little insight into effectiveness of specific tasks or types of tasks and it is for this reason that there is a need for micro-evaluation of individual task.

24. M ICRO – EVALUATIONS Micro evaluations is term for the evaluation that you conduct as part of your teaching-knowing your students and improving your teaching and their learning experiences.

25. K EY IDEAS OF MICRO - EVALUATIONS Micro-evaluations is about knowing your students Micro evaluation is primarily about the outcomes, processes and experiences of student learning. Micro evalution is a n integral part of teaching. Micro- evaluations involves cycles of reflection, data gathering, decision and action.

26. T YPES OF INFORMATION IN MICRO - EVALUATIONS Information regarding the learners opinion about the task. Information about how the task was performed. Information about what learning took place as a result of performing the task.

27. D IFFERENT APPROACHES TO EVALUATING TASK Student-based evaluation A response based evaluatio Learning based evaluation

28. M ICRO EVALUATION OF TASKS Elis (1998) outlined a procedure for conducting a micro evaluation of a task: 1. starting point is a description of a task in terms of its objectives. 2. the next step is to plan the evaluation by deciding on the objectives and purpose. the scope of the evaluation. who will conduct the evaluation. the timing. the types of information 3. the data for the evaluation are collected. 4. The data are analysed 5. Finally, conclusion and recommendations need to be made.

29. A NUMBER OF USES OF SUCH MICRO EVALUATION Simons(1997) evaluated an unfocused information-gap task. The task was performed in pairs it required to students A to describe a route marked on the map and student B could draw in the route on his or her map but the 2 maps were not identical and students’ A map included some information that was missing from student B’s map thus creating a number referential discrepancies. Simons’ aim was simply to establish whether the task was successful in eliciting meaningful communication

30. Five general categories were identified: 1. Telling 2. Questioning 3. Acknowledging 4. Responding 5. Miscellaneous Simons concluded because of the referential discrepancies of this information gap task was an effective device for including learners to use the second language communicating

31. T HE SECOND MICRO EVALUATION Freeman (2007) set out to evaluate a dicto_gloss task. This required students to a listening text three times I. First they were asked to answer a multiple choice question. II. The second, the students were told to note down the key content words. III. The third different students should focus and take notes on the use of different linguistic forms.

32. Freeman’s aims of evaluation was designed to establish both accountability and to provide information about how to improve the task so Freeman concluded that the students were largely successful in achieving the outcome of the task.

33. T HE THIRD TASK EVALUATION Conducted by Yuan (1997) examined two decision – making tasks (i.e. Unfocused tasks with open outcomes). He was interested in investigating the effect of implementation of variable – pre-task planning and the analysis focused on syntactical complexity, syntactical variety and lexical variety The results: an interesting finding is that the opportunity to plan did not affect the learner’s performance of the tasks in the same way. All three evaluations were concerned with establishing whether the tasks achieve what they were designed to achieve but only one (Freeman) also considered how the task might be improved. Only Freeman’s study included a student-based approach so it was the best among these evaluations

34. M ICRO BASED TASK - EVALUATION It is a good introspective opportunity for evaluating teaching techniques and materials.

35. CONCLUSION This chapter has examined the case for carrying out both macro- and micro-evaluations of the task-based teaching and has reported examples of both types.

36. A Publisher’s View

37. Overview Introduction Teacher vs Publisher Perspectives Why publisher evaluate materials? Methods Benefits and Limitation The future of Evaluation

38. Pedagogic vs business purposes teacher may review and refine the material at anytime, even for the next meeting. Publisher: once printed, the material are fixed for years Students’ familiarity/information Teachers can reflect on what worked well, disaster, or took too long due to having a lot of information Prepare material for unknown classess of students

39. Pedagogical o Material effectiveness, level of appropriateness, o Therefore, most ELT publishers and editors have a teaching background Business o Comercial attratctiveness, including the cost for developing it o It is also vital to keep their reputation

40. Methods Piloting Reviewing Focus group discussion Questionnaires Expert Panels Cooperation with academics and material developers on research project Editorial visits and classroom observation Desk research and competitor analysis

41. Piloters are sent a short selection material Using a separate teaching diary, annotate the unit pages Give comment on what went well, what went wrong, other comment like clarity of rubrics, activities, students’ questions, etc. Feedback is collated into a single pilot report Data analysis

42. Reviewing Reviewed by experienced teachers, academics, experts They are sent a small selection of materials They are sent a review sheet

43. Focus Group Face to face meeting with a facilitator (experienced editor) Ther is a general prompt question Adv: Two-way immediate interaction Allowing discussion between the participants Disadv: Can only done with a small number of people Sometimes, prefer to what the other member want to hear Can emerge “group think”

44. Questionnaires Cover a lot of ground with limited time and expenditure To be effective, it should be short and specific Expert Panels Selected panel of experts to review and give advise Cooperative with academic and material developers on research project Work with researcher in certain project

45. Editorial Visit Editorial Visits and Classroom Observation  Publisher send editors around the world to observe students  During piloting, classroom observation is also done Goal focus on the way the material is used whether the design is executed as planned Desk research and competitor analysis  visit the internet to see what is new  Access specialist sites

46. Who get the benefit?  Piloters/teachers  School  Publisher What can go wrong?  Minimum information from the reviewers  Reviewers tell what the publishers want to hear  The critics do not address/identify the problems  Ideas you are developing are found out by more people

47. Future of Evaluation Development in evaluation Technology engagement