1. Using the IDI to Measure the Impact of Saint Mary’s College Study Abroad Programs Elaine Meyer-Lee Saint Mary’s College NAFSA 2006 meyerlee@saintmarys.edu http://www.saintmarys.edu/~cwil/php/inter cultural.learning/IILOutcomes.php

2. Study Abroad at Saint Mary’s College Catholic Women’s Liberal Art (about 1600 students) Catholic Women’s Liberal Art (about 1600 students) 20% of students study abroad 20% of students study abroad 8 semester options 8 semester options 9 short-term options 9 short-term options Assessment Methods/Design Longitudinal Research Design Longitudinal Research Design Pre-test/Post-test (some grad follow-up)Pre-test/Post-test (some grad follow-up) Procedures – 70% response rate so farProcedures – 70% response rate so far Control Group Control Group 30 First-year students30 First-year students Focus Group Focus Group Study Abroad Re-entry CourseStudy Abroad Re-entry Course Written reflection and Kolb’s LSIWritten reflection and Kolb’s LSI

3. Slide Available Online:http://www.coe.int/T/E/North-South_Centre/Programmes/3_Global_Education/c_GENE/GENE%20Web- page%20-%20(3)%20Hesse%20article.asp

4. IDI Worldview Profile: Pre-Abroad

5. Significant findings from Pre-Post Match Data (n=60) Growth in Acceptance/Adaptation Scale on post test (0.01 level on paired T-tests) Growth in Acceptance/Adaptation Scale on post test (0.01 level on paired T-tests) Correlations: Correlations: Change in Minimization Scale-Universalism Cluster was correlated with specific program (0.05 level, see box plots and graph)Change in Minimization Scale-Universalism Cluster was correlated with specific program (0.05 level, see box plots and graph) Students on programs with more cultural immersion (4/5 vs. 3 on Engle & Engle’s scale) showed LESS growth in resolution on the Reversal Scale on average (0.01 level on paired T-tests)Students on programs with more cultural immersion (4/5 vs. 3 on Engle & Engle’s scale) showed LESS growth in resolution on the Reversal Scale on average (0.01 level on paired T-tests) Students with more previous experience abroad showed more growth in resolution on Reversal Scale on average (0.05 level)Students with more previous experience abroad showed more growth in resolution on Reversal Scale on average (0.05 level) Students with more years in college showed more growth in resolution on the Encapsulated Marginality Scale (0.05 level)Students with more years in college showed more growth in resolution on the Encapsulated Marginality Scale (0.05 level)

6. Box plot of Change in Minimization Scale by Program

8. Issues from Preliminary Results Given low Minimization scores (3 rd scale) Given low Minimization scores (3 rd scale) Catholic beliefs ….We’re all God’s children (universalism), orCatholic beliefs ….We’re all God’s children (universalism), or Underlying common humanity (similarity)Underlying common humanity (similarity) Does the IDI truly measure a developmental process? Does the IDI truly measure a developmental process? Is it legitimate to use an overall developmental score instead of subscales? Is it legitimate to use an overall developmental score instead of subscales?

9. Intercultural Development Inventory: Intercultural Development Inventory: Standardized instrument Standardized instrument Ability to compare study abroad program outcomes with domestic intercultural programs Ability to compare study abroad program outcomes with domestic intercultural programs ? Intercultural Development Inventory: ?Requires qualifying training to administer ?Highly proprietary ?Labor- and cost- intensive for large scale use Benefits to Study Abroad Assessment & Challenges to Institutional Research