An ESOL materials evaluation framework with a learning transfer focus Mark Andrew James Arizona State University

Summary Transfer of learning beyond the ESOL classroom is a basic goal of ESOL instruction. ESOL materials can be designed to support this goal in a number of ways. This presentation describes and demonstrates a practical tool for evaluating existing ESOL materials (e.g., commercial textbooks) from a learning transfer perspective.Transfer of learning beyond the ESOL classroom is a basic goal of ESOL instruction. ESOL materials can be designed to support this goal in a number of ways. This presentation describes and demonstrates a practical tool for evaluating existing ESOL materials (e.g., commercial textbooks) from a learning transfer perspective.

Why evaluate TESOL materials from a learning transfer perspective? Materials are pervasive and influential in TESOL, so evaluating them is worthwhile.Materials are pervasive and influential in TESOL, so evaluating them is worthwhile.... and textbooks may be the most common kind (Richards, 2001). They are “an almost universal element of ELT” (Rubdy, 2003, p.37), and are “perceived by many to be the route map of any ELT programme” (Sheldon, 1988, p.238).... and textbooks may be the most common kind (Richards, 2001). They are “an almost universal element of ELT” (Rubdy, 2003, p.37), and are “perceived by many to be the route map of any ELT programme” (Sheldon, 1988, p.238).... textbooks may be leaned on heavily by busy teachers, novice teachers, and students working independently.... textbooks may be leaned on heavily by busy teachers, novice teachers, and students working independently. Materials evaluation frameworks are available, but have gaps.Materials evaluation frameworks are available, but have gaps.... frameworks focus on factors like ease of navigation, attractiveness, potential to motivate, lower anxiety, stimulate student interaction (e.g., Reinders & Lewis, 2006; Tomlinson, 2011).... frameworks focus on factors like ease of navigation, attractiveness, potential to motivate, lower anxiety, stimulate student interaction (e.g., Reinders & Lewis, 2006; Tomlinson, 2011).... but, they don’t focus on learning transfer.... but, they don’t focus on learning transfer.

Why evaluate TESOL materials from a learning transfer perspective? Learning transfer is a goal of TESOL.Learning transfer is a goal of TESOL.... it is the process by which ”something learned in one context has helped in another” (Perkins & Salomon, 1988, p.22).... it is the process by which ”something learned in one context has helped in another” (Perkins & Salomon, 1988, p.22).... learning doesn’t inevitably transfer, and can be difficult to promote (Haskell, 2001).... learning doesn’t inevitably transfer, and can be difficult to promote (Haskell, 2001). TESOL materials can be designed to promote transfer.TESOL materials can be designed to promote transfer.... teaching-for-transfer techniques have been suggested for use by teachers in general education (Fogarty et al, 1992) and in TESOL (James, 2006).... teaching-for-transfer techniques have been suggested for use by teachers in general education (Fogarty et al, 1992) and in TESOL (James, 2006).... these techniques can be reflected in TESOL materials.... these techniques can be reflected in TESOL materials.

What does the framework look like? Main question 1: Does the textbook make connections to situations where students can apply what they have learned? (i.e., hugging)Main question 1: Does the textbook make connections to situations where students can apply what they have learned? (i.e., hugging) Main question 2: Does the textbook get students to make conscious abstractions and seek applications of learning? (i.e., bridging)Main question 2: Does the textbook get students to make conscious abstractions and seek applications of learning? (i.e., bridging) (“Bridging” and “hugging” come from Perkins & Salomon, 1988.)

What does the framework look like? Main question 1: Does the textbook make connections to situations where students can apply what they have learned? (i.e., hugging)Main question 1: Does the textbook make connections to situations where students can apply what they have learned? (i.e., hugging) More specifically, how (if at all) does the textbook...More specifically, how (if at all) does the textbook... tell students that learning can be applied outside ESOL classrooms? (i.e., setting expectations)tell students that learning can be applied outside ESOL classrooms? (i.e., setting expectations) get students to work with (i.e., read/listen, produce) English texts that occur outside ESOL classrooms? (i.e., matching)get students to work with (i.e., read/listen, produce) English texts that occur outside ESOL classrooms? (i.e., matching) get students to use English while pretending to be in situations outside ESOL classrooms? (i.e., simulating)get students to use English while pretending to be in situations outside ESOL classrooms? (i.e., simulating) demonstrate the use of English? (i.e., modeling)demonstrate the use of English? (i.e., modeling) get students to use English to solve problems that occur outside ESOL classrooms? (i.e., problem-based learning)get students to use English to solve problems that occur outside ESOL classrooms? (i.e., problem-based learning)

What does the framework look like? Main question 2: Does the textbook get students to make conscious abstractions and seek applications of learning? (i.e., bridging)Main question 2: Does the textbook get students to make conscious abstractions and seek applications of learning? (i.e., bridging) More specifically, how (if at all) does the textbook get students to...More specifically, how (if at all) does the textbook get students to... look for situations where learning can be applied outside ESOL classrooms? (i.e., anticipating applications)look for situations where learning can be applied outside ESOL classrooms? (i.e., anticipating applications) look for general rules, principles, or patterns behind English use? (i.e., generalizing concepts)look for general rules, principles, or patterns behind English use? (i.e., generalizing concepts) look for similarities between English use and something else they know? (i.e., using analogies)look for similarities between English use and something else they know? (i.e., using analogies) use English to solve multiple problems that are superficially different but structurally similar? (i.e., parallel problem solving)use English to solve multiple problems that are superficially different but structurally similar? (i.e., parallel problem solving) plan, monitor, and reflect on their use of English? (i.e., metacognitive reflection)plan, monitor, and reflect on their use of English? (i.e., metacognitive reflection)

How does the framework work? Demonstrate by comparing two textbooks, for adults at the low intermediate level:Demonstrate by comparing two textbooks, for adults at the low intermediate level: Four corners, student book 2 (Richards & Bohlke) Network, student book 2 (Hutchinson & Sherman)

1. How (if at all) does the textbook tell students that learning can be applied outside ESOL classrooms? (i.e., setting expectations) Four corners: pictures of people talking during a video conference, and in a kitchen.pictures of people talking during a video conference, and in a kitchen. Network: pictures of people talking during a phone call, and in a cafeteria.pictures of people talking during a phone call, and in a cafeteria.

2. How (if at all) does the textbook get students to work with (i.e., read/listen, produce) English texts that occur outside ESOL classrooms? (i.e., matching) Four corners: listening and speaking in conversations;listening and speaking in conversations; listening to conversations;listening to conversations; reading a tourist brochure, an , and the textbook;reading a tourist brochure, an , and the textbook; writing an .writing an .Network: listening and speaking in conversations;listening and speaking in conversations; listening to conversations and to a tv report;listening to conversations and to a tv report; reading a magazine article and the textbook.reading a magazine article and the textbook.

3. How (if at all) does the textbook get students to use English while pretending to be outside ESOL classrooms? (i.e., simulating) Four corners: pretend to be looking out a window, and decide with a partner what to do outside;pretend to be looking out a window, and decide with a partner what to do outside; pretend to be making summer plans with a friend, and write an to her/him.pretend to be making summer plans with a friend, and write an to her/him. Network: pretend one partner is a character from an article, and interview him.pretend one partner is a character from an article, and interview him.

4. How (if at all) does the textbook demonstrate the use of English? (i.e., modeling) Four corners: listen to recordings of conversations or sentences, then repeat;listen to recordings of conversations or sentences, then repeat; read conversations showing people comparing answers, making plans together, asking for more information, or guessing, then have a conversation with same purpose;read conversations showing people comparing answers, making plans together, asking for more information, or guessing, then have a conversation with same purpose; listen to recordings of people talking about seasons or the best time to visit cities, then talk about the same thing;listen to recordings of people talking about seasons or the best time to visit cities, then talk about the same thing; read , then write one.read , then write one. Network: listen to recordings of conversations or sentences, then repeat;listen to recordings of conversations or sentences, then repeat; read conversations showing people asking each other about jobs, or guessing, or interviewing, then have a conversation with same purpose.read conversations showing people asking each other about jobs, or guessing, or interviewing, then have a conversation with same purpose.

5. How (if at all) does the textbook get students to use English to solve problems that occur outside ESOL classrooms? (i.e., problem-based learning*) Four corners: guess the name of a place a partner is describing;guess the name of a place a partner is describing; guess which of a partner’s sentences about hometown are false;guess which of a partner’s sentences about hometown are false; find out if one’s opinion is same as partner’s;find out if one’s opinion is same as partner’s; agree on future plans with a partner;agree on future plans with a partner; Network: guess what job a partner is thinking of.guess what job a partner is thinking of. * Mathews-Aydinli (2007) suggested that PBL in TESOL should involve not just getting information but using it to make decisions and judgments.

6. How (if at all) does the textbook get students to look for situations where learning can be applied outside ESOL classrooms? (i.e., anticipating applications) Four corners: listen to a conversation and answer the open-ended question, “Where are the people?”;listen to a conversation and answer the open-ended question, “Where are the people?”; read an article and answer the multiple-choice question “Where is it from?” (e.g., tourist brochure, vacation blog).read an article and answer the multiple-choice question “Where is it from?” (e.g., tourist brochure, vacation blog). Network:---

7. How (if at all) does the textbook get students to look for general rules, patterns, or principles behind English use? (i.e., generalizing concepts) Four corners: match weather words with pictures of weather;match weather words with pictures of weather; match game words with pictures of games.match game words with pictures of games. Network: match job words with definitions.match job words with definitions.

8. How (if at all) does the textbook get students to look for similarities between English use and something else they know? (i.e., using analogies) Four corners: --- Network:---

9. How (if at all) does the textbook get students to use English to solve multiple problems that are superficially different but structurally similar? (i.e., parallel problem solving) Four corners: guess about different things a partner is talking about (i.e., the weather in her/his country, a famous place in her/his country);guess about different things a partner is talking about (i.e., the weather in her/his country, a famous place in her/his country); with a partner, agree on plans for different occasions (i.e., today, this weekend).with a partner, agree on plans for different occasions (i.e., today, this weekend). Network:---

10. How (if at all) does the textbook get students to plan, monitor, and reflect on their use of English? (i.e., metacognitive reflection) Four corners: compare answers with a partner (i.e., grammar and vocab exercises);compare answers with a partner (i.e., grammar and vocab exercises); check answers by listening to a recording again;check answers by listening to a recording again; complete self-evaluation statements (e.g., “I can talk about a place I’d like to visit”);complete self-evaluation statements (e.g., “I can talk about a place I’d like to visit”); brainstorm a list of words (i.e., weather) to see how many one knows;brainstorm a list of words (i.e., weather) to see how many one knows; identify questions one can ask to get opinions.identify questions one can ask to get opinions.Network: compare answers with a partner (i.e., listening comprehension exercises);compare answers with a partner (i.e., listening comprehension exercises); check answers by listening to a recording again;check answers by listening to a recording again; complete self-evaluation statements (e.g., “I can talk about a place I’d like to visit”).complete self-evaluation statements (e.g., “I can talk about a place I’d like to visit”).

Summary Both of these textbooks reflect a range of hugging techniques and a range of bridging techniques.Both of these textbooks reflect a range of hugging techniques and a range of bridging techniques.... but, there are also gaps in both:... but, there are also gaps in both:... no explicit setting expectations;... no explicit setting expectations;... no using analogies;... no using analogies;... modeling is only for speaking and writing;... modeling is only for speaking and writing;... generalizing concepts is only for vocabulary.... generalizing concepts is only for vocabulary. And there are differences between the two units:And there are differences between the two units:... Four corners has a relatively broad range for half the techniques (i.e., metacognitive reflection, parallel problem solving, problem based learning, modeling, and matching), and includes some anticipating applications, and matching for writing skills.... Four corners has a relatively broad range for half the techniques (i.e., metacognitive reflection, parallel problem solving, problem based learning, modeling, and matching), and includes some anticipating applications, and matching for writing skills.

Conclusion This materials evaluation framework can provide useful information from a small sample of materials (e.g., 1 textbook unit); but, a small sample may mean other useful information is missed.This materials evaluation framework can provide useful information from a small sample of materials (e.g., 1 textbook unit); but, a small sample may mean other useful information is missed. This evaluation framework shows how materials reflect teaching-for- transfer techniques; but, reflecting these techniques does not necessarily lead to transfer.This evaluation framework shows how materials reflect teaching-for- transfer techniques; but, reflecting these techniques does not necessarily lead to transfer..... research that looks at the relationship between (a) teaching-for- transfer techniques in materials and (b) actual transfer would be worthwhile..... research that looks at the relationship between (a) teaching-for- transfer techniques in materials and (b) actual transfer would be worthwhile.... for example, do materials that reflect more techniques lead to more transfer? Or more of a certain kind of technique? Or certain combinations of techniques? Or certain kinds of activities for a particular technique?... for example, do materials that reflect more techniques lead to more transfer? Or more of a certain kind of technique? Or certain combinations of techniques? Or certain kinds of activities for a particular technique?

References Fogarty, R., Perkins, D., & Barell, J. (1992). How to teach for transfer. Palatine: Skylight.Fogarty, R., Perkins, D., & Barell, J. (1992). How to teach for transfer. Palatine: Skylight. Haskell, R.E. (2001). Transfer of learning. San Diego, CA: Academic Press.Haskell, R.E. (2001). Transfer of learning. San Diego, CA: Academic Press. Hutchinson, T., & Sherman, K. (2012). Network (student book 2). Oxford, UK: Oxford University Press.Hutchinson, T., & Sherman, K. (2012). Network (student book 2). Oxford, UK: Oxford University Press. James, M.A. (2006). Teaching for transfer in ELT. ELT Journal, 60, James, M.A. (2006). Teaching for transfer in ELT. ELT Journal, 60, Mathews-Aydinli, J. (2007). Problem based learning and adult English language learners. Washington, DC: CAELA.Mathews-Aydinli, J. (2007). Problem based learning and adult English language learners. Washington, DC: CAELA. Perkins, D.N., & Salomon, G. (1988). Teaching for transfer. Educational Leadership, 46, Perkins, D.N., & Salomon, G. (1988). Teaching for transfer. Educational Leadership, 46, Reinders, H., & Lewis, M. (2006). An evaluative checklist for self-access materials. ELT Journal, 60, Reinders, H., & Lewis, M. (2006). An evaluative checklist for self-access materials. ELT Journal, 60, Richards, J.C. (2001). Curriculum development in language teaching. Cambridge, UK: Cambridge University Press.Richards, J.C. (2001). Curriculum development in language teaching. Cambridge, UK: Cambridge University Press. Richards, J.C., & Bohlke, D. (2012). Four corners (student book 2). Cambridge, UK: Cambridge University Press.Richards, J.C., & Bohlke, D. (2012). Four corners (student book 2). Cambridge, UK: Cambridge University Press. Rubdy, R. (2003). Selection of materials. In B. Tomlinson (Ed.), Developing materials for language teaching. London, UK: Continuum.Rubdy, R. (2003). Selection of materials. In B. Tomlinson (Ed.), Developing materials for language teaching. London, UK: Continuum. Sheldon, L.E. (1988). Evaluation of ELT textbooks and materials. ELT Journal, 42, Sheldon, L.E. (1988). Evaluation of ELT textbooks and materials. ELT Journal, 42, Tomlinson, B. (Ed.) (2011). Materials development in language teaching (2 nd ed.). Cambridge, UK: Cambridge University Press.Tomlinson, B. (Ed.) (2011). Materials development in language teaching (2 nd ed.). Cambridge, UK: Cambridge University Press.

... thank you! Feel free to contact me with any remaining questions/comments: Presentation recording and slides can be downloaded at: