Intercultural Competence Conference January 24, 2014 The Intersection of Language & Culture in Study Abroad: Assessment & Analysis of SA Outcomes Intercultural Competence Conference January 24, 2014 Dr. Jeff Watson jeffrey.watson@usma.edu Dr. Richard Wolfel richard.wolfel@usma.edu

Agenda Interconnectedness of Language and Culture in Study Abroad What makes study abroad so rewarding? Assessing SA Outcomes from a L2 Socialization Perspective Language Proficiency and Intercultural Competence Socialization Variables Study Method & Results Conclusions & Implications

Interconnectedness of Language & Culture in Study Abroad DEFINITION “…extraordinarily difficult and elusive task…” (Kasper & Omori, 2007) LINK “…language expresses, embodies, and symbolizes cultural reality” (Kramsch, 1998) Language plays a role in the genesis of society (Watson, 2010) and is an intricate part of the “cultural fabric” within which “language is shaped and meanings are produced” (Duranti, 2009).

The “magic” of Study Abroad SA provides a highly contextualized learning environment with virtually limitless access to native speech communities, authentic sociocultural behaviors, and identity-destabilizing experiences (Kinginger, 2008) SA participants interact with this environment in ways unique to their own interests, motivations, and awarenesses and teach them “to think, feel, and act in accordance with the values, ideologies, and traditions” of their target community (Duff, 2008)

Assessing SA Outcomes from a L2 Socialization Perspective SA outcomes are: -complex, multidimensional, interrelated -involve social interaction and the degree in which SA participants involve themselves in it - typically “contextualized within particular routine activities” (Duff, 2008)

How & what do we assess? L2 Proficiency (Listening, Reading, Speaking) Defense Language Proficiency Test (DLPT) Oral Proficiency Interview (OPI) Intercultural Competence Intercultural Development Inventory (IDI) Socialization Variables Lodging Type: Dormstay vs. Homestay Interaction Time-on-task (self-report survey) - Amount of English Spoken - Amount of TL Conversation with Native Speakers - Amount of Cultural Activities

Internationalization of the Curriculum: “Crawl, Walk, Run” Developmental Path for Language Proficiency, Cross-Cultural Competence, & Regional Awareness Advanced Language 2LT Semester Abroad Summer Immersion RUN Intermediate Language WALK Spring Immersion Semester CRAWL Basic Language Summer Spring Break

Population & Research Design Population: 279 semester-abroad participants - AY 2011-2013 - juniors, seniors (self-selected) - 225 males; 54 females - 32 host institutions; 22 cities; 15 countries - Languages: Group 1: MSA, Mandarin, Russian Group 2: French, German, Spanish, Portuguese -Lodging: 176 dormstay; 103 homestay Research Design: Pre- and post-immersion testing

Data Collection & Analysis Pre- and Post-Immersion Seminars & Assessments: Seminars: Culture Shock & Experience Processing, Maximizing SA Focus Groups: Regional Awareness, ICC, Cultural Comparisons, Identity Shifts, Stereotypes Language Assessments: DLPT & OPI Culture Assessments: IDI & MPQ Regional Assessments: ARK & RDT Experience Processing Survey Motivation/Learning Strategy Use Abroad Time-on-task Survey

Arabic, Chinese, Russian French, German, Portuguese, Spanish Gains in L2 Proficiency Average gains: one half-step increase in proficiency after one semester.   Group 1: Arabic, Chinese, Russian Group 2: French, German, Portuguese, Spanish Wilcoxon Ranks Cohen’s d Listening Z=-6.88, p=.001 d=-0.64 (med-large) Z=-8.20, p=.001 d=-1.29 (large) Reading Z=-7.76, p=.001 d=-0.89 (large) Z=-7.53, p=.001 d=-1.00 (large) Speaking Z=-8.52, p=.001 d=-0.97 (large) Z=-8.24, p=.001 d=-1.09 (large)

Gains in Intercultural Competence 55-85 85-115 115-130 55-85 85-115 115-130

Gains in Intercultural Competence   Group 1: Arabic, Chinese, Russian Group 2: French, German, Portuguese, Spanish t-test Cohen’s d IDI M=5.53 SD=12.11 t(95)=4.47 p=.001 d = .917 (large) M= 3.85 SD=12.10 t(78)=2.83 p=.006 d = .640 (medium)

Socialization Variables Group 1 Average Group 2 Average Lodging type: Dormstay: 104 (74.3%) Homestay: 36 (25.7%) Dormstay: 72 (51.8%) Homestay: 67 (48.2%) Percentage of English used: (inverse relationship with gains expected) 41% 28% Time spent in TL conversation: average hours per week 17.9 hrs./week 33.6 hrs./week Participation in Cultural Activities: average hours per week spent in cultural activities outside of class (e.g., tours, excursions, attending sporting events, clubs, church, community service) 13.6 hrs./week 18.1 hrs./week

Lodging Type Analysis: Group 1 2011-present

Lodging Type Analysis: Group 2 2011-present

Average hours of TL Conversation Average hours of Cultural Activities Correlations   Listening Gain Reading Gain Speaking Gain ICC Gain Amount of English used Average hours of TL Conversation Average hours of Cultural Activities r Group 1 1.00 .416** .329** -0.11 -.295* 0.17 0.04 Group 2 .301** 0.06 0.15 0.00 .311** .287** -0.08 0.08 0.07 .188* 0.20 0.21 -0.02 -0.13 -.388** -0.04 -0.21 -0.06 0.05 IC Gain 0.16 0.22 .386** -.363** 0.01 -0.24 -0.12 .257* .508**

Discussion Gains in L2 Proficiency don’t necessarily affect gains in intercultural competence. (Ouch!) One semester abroad may not be enough to see a meaningful relationship between these two variables. Lodging type remains an important variable but perhaps not as important as the type(s) of interaction available in each type. Participation in cultural activities outside of class seems to play a role in gaining intercultural competence but not always. Why? Participation in cultural activities outside of class seems to provide opportunities for conversation in the target language but not always. Amount of English spoken abroad seems to affect gains in Listening & Speaking but not always. Why?

Implications What specific cultural activities abroad play a role in increasing intercultural competence? What specific cultural activities abroad provide opportunities for TL interaction and play a role in increasing L2 proficiency? What types of TL interactions are typical of and most meaningful in dormstay vs. homestay contexts? In which context(s) is English used by SA participants abroad? How can we encourage more TL use? What individual differences (strategies, motivation, characteristics) are evident in SA participants abroad? Which types of individual differences are most meaningful for gains in L2 proficiency and IC competence? Which during-immersion activities by SA participants will provide useful qualitative data and further reinforce language and culture learning abroad?

Evidence-based Suggestions for Program Administration pre-departure assessment analysis to “match” students with a lodging situation or institution abroad most conducive to their strengths teach strategies pre-departure for increasing language contact during cultural activities or when navigating daily life in a host family or dormitory environment work with host institutions to provide/encourage structured language contact opportunities outside of class such as community service, peer coaching, travel or family activities, along with reflection exercises that can be used for mentoring, reinforcement of program goals, and benefit analysis. Use student performance data to assess the effectiveness of SA host institutions and improve goal alignment with home program

Intercultural Competence Conference January 24, 2014 The Intersection of Language & Culture in Study Abroad: Assessment & Analysis of SA Outcomes Intercultural Competence Conference January 24, 2014 Dr. Jeff Watson jeffrey.watson@usma.edu Dr. Richard Wolfel richard.wolfel@usma.edu