Presentation is loading. Please wait.

Presentation is loading. Please wait.

Evaluation of the UF Summer Abroad Program Florida Consortium for International Education David Sammons Dena Roberts James Demery October 6, 2011 Orlando,

Similar presentations


Presentation on theme: "Evaluation of the UF Summer Abroad Program Florida Consortium for International Education David Sammons Dena Roberts James Demery October 6, 2011 Orlando,"— Presentation transcript:

1 Evaluation of the UF Summer Abroad Program Florida Consortium for International Education David Sammons Dena Roberts James Demery October 6, 2011 Orlando, Florida 1

2  Program Improvement ◦ Satisfaction with program services and facilities ◦ Satisfaction with the experience  Program Impact ◦ Changes in attitudes, skills, behaviors  Changes that occurred:  while students were abroad  because they were abroad 2

3 3 Inputs Resources ActivitiesOutputs Immediate Outcome Long-term Impact Applicants Student fees Annual budget Staff (10 fulltime and 1 part-time) Facilities Faculty support Information sessions Recruitment and placement Faculty and student workshops Staff training Students studying abroad Cross-cultural faculty teaching Research partnerships Publications Presentations Clear career aspirations Increased self- awareness Willing to take appropriate risks Respect for diversity Increased intercultural learning Global citizenship: Behave in ways that sustain and enhance the economic, political, social, environmental systems of the global community

4  How did studying abroad impact the intercultural learning and global citizenship of UF students?  What impact does gender, academic program, class level, ethnicity, parents’ educational attainment, and political affiliation have on intercultural learning and global citizenship?  How can the UF Summer Study Aboard program be improved? 4

5  Mixed-method ◦ Surveys  Study Abroad and Non-study Abroad ◦ Interviews  Study Abroad students ◦ Document reviews 5

6  Quasi-experimental study ◦ Nonequivalent pretest-posttest control group  Matching vs. random assignment  Limitations of the study ◦ Self-report data ◦ Retrospective Pretest for the ILO survey ◦ Size of the control group (n=50) for ILO survey limited the minimum detectable effect to.30 6

7  Intercultural Learning Objectives survey (ILO) ◦ 1001 Recipients, 494 Respondents (Study Abroad) ◦ 60 Recipients, 50 Respondents (Control group)  UF Study Abroad Services survey (SAS) ◦ 1001 Recipients, 494 Respondents ◦ *Administered ILO and SAS surveys at same time 7

8  Student Experience in Research Universities survey (SERU) ◦ 455 of the 494 ILO and SAS respondents ◦ 455 of the 15,000 Non-SA students (Control)  UF Study Abroad Services interview ◦ 15 of the 494 SAS respondents 8

9  Phase Two (all Study Abroad terms) ◦ Global Competence Awareness Assessment (GCAA) 9

10  Avoid Type I errors (false positives) and Type II errors (false negatives)  Optimal Design Software (Univ. of Michigan) ◦ Alpha level (.05); null hypothesis rejection ◦ Power (.80); to detect meaningful effects ◦ 50 students in control group *Minimum detectable effect on ILO:.30 std. dev. 10

11  T-tests for significant difference in means ◦ Paired sample t-tests (pretest vs. posttest) ◦ Independent samples t-tests (SA vs. Control)  ANOVA (with Bonferroni correction) ◦ Where are the significant differences?  Chi square 11

12 12 Composition of ILO and SAS survey Recipients and Respondents

13 13 Ethnic Composition of ILO and SAS Recipients and Respondents

14 College Recipients (N=1001) Respondents (n=494) Accounting1% Agriculture & Life Science17%8% Business15%11% Design, Construct, & Planning1%5% Education2% Engineering7% Fine Arts5%3% Health & Human Performance5%4% Journalism & Communication12% Law<1%2% Liberal Arts & Sciences31%30% Natural Resources & Environ.<1%1% Nursing<1% Public Health<1%1% Undecided2%<1% ILO and SAS Survey Recipients and Respondents by College 14

15 ILO Survey and SAS Survey Respondents by Political Affiliation 15

16 ILO Survey and SAS Survey Respondents by Class Level 16

17 Educational Attainment Mother (n = 494) Father (n = 494) No formal education< 1% Less than HS diploma3%6% High school diploma18%17% Assoc./Postsecondary cert.12%17% Bachelor’s34%24% Post-baccalaureate cert.< 1% Master’s15%17% Professional5%8% Doctoral3%9% Did not indicate10% Parental Education: ILO Survey and SAS Survey Respondents 17

18  Five Domains ◦ Functional Knowledge ◦ Knowledge of World Geography ◦ Knowledge of Global Interdependence ◦ Knowledge of Interpersonal Accommodation ◦ Knowledge of Cultural Sensitivity 18

19  Lead discussions and conversations  Talk way out of difficult situations  Give directions  Pacify an angry person  Know what’s funny  Know advantage of taking risks  Locate information in newspaper 19

20  Locate a bar or restaurant  Take public transportation  Use public telephone  Purchase toothpaste  Know norms and taboos  Know different ways of expressing ideas  Can compare languages and cultures 20

21  Can name 4 rivers n Europe  Can name 3 rivers in Asia  Can name 6 countries in Africa  Can name 4 countries in South America  Can name the 7 continents 21

22  Know how world markets affect own career  Can explain US foreign policy  Know how foreign manufacturing affects costs of goods in the US  Understand US military situation (N. Korea)  Can compare political and cultural freedom 22

23  Know importance of being flexible  Know importance of being patient 23

24  Understand own reactions in different situations  Understand significance of language and cultural differences  Know how settings affect own style of interacting 24

25  Student satisfaction with: ◦ Staff ◦ Services (UF and abroad) ◦ Facilities (UF and abroad) ◦ Housing ◦ Excursions ◦ Host country  Suggestions for improvement 25

26  Deeper probe of SAS survey responses  Triangulation (i.e., reliability check) 26

27 Developed by UC, Berkeley  Mission of SERU ◦ to help broaden our understanding of the undergraduate experience and to promote a culture of institutional self-improvement  At UF, approximately 20,000 students participate 27

28

29 Increased linguistic competency Defined Career aspirations Increased self-awareness Willingness to take risks Global awareness and knowledge Intercultural competence Cross-cultural engagement Respect for diversity Enhanced communication skills Personal Growth Improved learning skills International Job skills Personal expectations met Continued global competency

30 Construct/DomainN Mean (Before) Mean (After) Mean GainStd. Dev. Sig. (2-tailed) Intercultural Learning Functional Knowledge World Geography Global Interdependence Interpersonal Accommodation Cultural Sensitivity Changes in Intercultural Learning as Measured by the ILO Survey (Study Abroad Participants)

31  Very liberals (M = 4.27, SD =.491) and conservatives (M = 3.92, SD =.658).  Very liberal (M = 4.27, SD =.491) and moderates (M= 3.93, SD =.551).

32  There a statistically significant gain in functional knowledge by political affiliation between:  Very liberals (M = 4.20, SD =.534) and conservatives (M = 3.87, SD =.703).

33  There a statistically significant gain in world geography by political affiliation between:  Very liberals (M = 4.20, SD =.829) and slightly liberals (M = 3.66, SD =.846).  Very liberals (M = 4.20, SD =.829) and moderates (M = 3.86, SD =.729).  Very liberals (M = 4.20, SD =.829) and conservatives (M = 3.77, SD =.831).

34  There a statistically significant gain in global interdependence by political affiliation between:  Very liberals (M = 4.14, SD =.669) and moderates (M = 3.75, SD =.754).

35  There a statistically significant gain in interpersonal accommodation by college between:  College of Liberal Arts and Sciences (M = 4.44, SD =.625) and Design Construction Planning (M = 3.91, SD =.900).

36  There a statistically significant gain in cultural sensitivity by college between:  College of Liberal Arts and Science (M = 4.45, SD =.548) and Engineering (M = 3.96, SD =.848).  There was also a statistically significant gain in cultural sensitivity by political affiliation between:  Very liberals (M = 4.53, SD =.528) and moderates (M = 4.09, SD =.707).  Very liberals (M = 4.53, SD =.528) and conservatives (M = 4.12, SD =.823).

37 Construct/Domain Control Group Study Abroad Mean Diff. Std. Dev. Sig. (2- tailed) Effect Size Percentile Gains Intercultural Learning th to 88 th Functional Knowledge th to 92 nd World Geography th to 76 th Global Interdependence th to 75 th Interpersonal Accommodation th to 60 th Cultural Sensitivity th to 76 th

38 YesNo Internship under the direction of a faculty member Other internship (e.g., co-op, clinical assignment) Any UF study abroad program A study abroad program with another college or university Have you completed or are you now participating in any of the following activities?

39 When you started at this campus, and Now Very Poor PoorFairGood Very Good Excellen t Foreign language skills Please rate your level of proficiency in the following areas when you started here and now.

40 When you started here, and now Very Poor PoorFairGood Very Good Excellent Linguistic and cultural competency in at least one language other than my own. How would you rate your competencies in the flowing areas when you started here and now?

41  National Certification ◦ (for some programs)

42 How much did you improve your ability to communicate in your host country’s language while abroad?

43 Do you plan to continue your language studies at UF or at your home institution?

44  What do you plan to do when you graduate? ◦ Enroll in graduate or professional school ◦ Work full time ◦ Be self-employed ◦ Study or work abroad ◦ Join armed forces ◦ Work in an internship or volunteer position ◦ Take a year off ◦ Do something else ◦ I have no idea

45  What do you plan to do when you graduate? ◦ Enroll in graduate or professional school ◦ Work full time ◦ Be self-employed ◦ Study or work abroad ◦ Join armed forces ◦ Work in an internship or volunteer position ◦ Take a year off ◦ Do something else ◦ I have no idea

46  What is the highest academic degree or credential you plan to earn, eventually? ◦ Bachelor’s degree (BA, BS, etc.) ◦ Teaching credential ◦ Business mater’s (MBA) ◦ Other professional master’s (MEd, MPP, MPH, MFA, MLIS, MSN, MSW, M.ARCH) ◦ Academic master’s (MA, MS, etc.) Law Degree (LLB or JD) ◦ Medical doctorate other than MD (DO, DDS, DVM, etc.) ◦ Medical doctorate (MD) Doctorate (PhD, EdD, etc.) ◦ Multiple doctoral degrees (MD/PhD) ◦ I don’t know yet

47 Did the study abroad experience influence you to change your major or minor?

48  Do you have different academic goals after going abroad?  85% of the summer and spring 2011 students said “yes.”

49  Would you like to work in your host country after graduation? ◦ 90% said yes

50 Has study abroad influenced your career interests?

51 As a University of Florida student, how would you rate your competencies below?

52  Do you think studying aboard makes students better international job candidates? ◦ 100% said “YES”

53  Do you think you’d like to work in your host country after graduation? ◦ 90% said YES ◦ No, because they wanted something different or it was difficult to get jobs in that country without family connections.

54 Strongly disagree Disagree Somewhat disagree Somewhat agree Agree Strongly agree I obtained the skills I need to function in the international arena I obtained information about international opportunities in my academic field I increased my knowledge about the ethics, culture, politics, society or values of another country Please rate your level of agreement with the following statements about your experience at UF.

55 NeverRarelyOccasionally Somewhat often Often Very often Worked with a faculty member on a project with an international/global theme Presented a paper at a symposium or conference or participated in a panel on international/global topics Attended lectures, symposia, workshops or conferences on international/global topics While attending UF how frequently have you engaged in the following?

56

57 Strongly disagree Disagree Somewhat disagree Somewhat agree Agree Strongly agree I have a deeper understanding of another country’s culture based on activities on campus I evaluated my own values based on another culture I increased my knowledge about the ethics, culture, politics, society or values of another country Please rate your level of agreement with the following statements about your experience at UF.

58 NeverRarelyOccasionally Somewhat often Often Very often Their religious beliefs were very different from my own Their political opinions were very different from my own They were an immigrant or from an immigrant family They were of a different nationality than my own They were of a different race or ethnicity than my own Their gender was different Their sexual orientation was different They were from a different social class They had physical or other observable disabilities They had learning, psychological, or other disabilities that are not readily apparent How often have you gained a deeper understanding of other perspectives through conversations with fellow students because they differed from you in the following ways?

59  Ability to recognize individual and cultural differences between the host country and USA ◦ Religious freedom ◦ Political freedom ◦ public safety ◦ Relationships? Dating/courtship rituals? ◦ Leisure activities  100% of all of students could speak about some kind of similarities or differences.

60  How did the host country’s teaching style(s) affect your learning? - Summer programs, so there are mostly UF professors, so they were not exposed to new teaching styles

61  Did the teaching styles match your preferred learning style? - 50% mixed answers -No, Yes

62 NeverRarelyOccasionally Somewhat often OftenVery often The United States Countries outside the US Global politics and diplomacy Global climate and environmental issues International business and economics Global health issues During this academic year how frequently have you followed news about the following?

63 Did you fulfill your expectations of your program?

64 Would you recommend this program to other students? 92.2% Yes 7.8% No

65 How does the cost of studying abroad compare to the benefits?

66  Would you say this experience made you a better person or are about the same as before?  91% yes  9% Same

67 Not important Somewhat important Very important Be in a position to give something back to my community after finishing my education Discover what kind of person I really want to be Explore new ideas Indicate how important each of the following college goals is to you.

68 Strongly disagree Disagree Somewhat disagree Somewhat agree Agree Strongly agree I increased my knowledge about the ethics, culture, politics, society or values of another country Please rate your level of agreement with the following statements about your experience at UF.

69  With the collected data, we want to disseminate the information to our stakeholders. ◦ Students, faculty, administrators, staff and parents  Create reports for faculty led programs to improve their study abroad programs  Marketing purposes  Justify our budget and increase the value of a student’s University of Florida degree


Download ppt "Evaluation of the UF Summer Abroad Program Florida Consortium for International Education David Sammons Dena Roberts James Demery October 6, 2011 Orlando,"

Similar presentations


Ads by Google