COMPARATIVE ASSESSMENT OF SCALED GLOBAL ENGINEERING INITIATIVES Diane DeTurris, Jane Lehr, Jim Widmann, Lily Hsu Laiho, Fred DePiero, Zoë Wood, Alana Snelling California Polytechnic State University San Luis Obispo, CA ASEE International Forum 14 June 2014

What is Global Competency for Engineers? Knowledge: geographic, geopolitical, historical, cultural, social, economic, etc. Including analysis of one’s own country and culture of origin Skills: Written and oral communication (for multiple audiences and stakeholders), multi-lingualism, teamwork, self-reflexivity, leadership, critical thinking, project management, ability to cope with unfamiliarity and difference and to understand the ways in which these impact behaviors and practices and then modify social and technical practices to work effectively in multiple national contexts, etc. Attitudes: Openness, willingness to participate in and learn from other cultures, anti-ethnocentricism, etc.

What is Global Competency for Engineers? Gary Downey et al (2006), The Globally Competent Engineer: Working Effectively with People Who Define Problems Differently. Journal of Engineering Education, 95(2): … students will acquire the knowledge, ability, and predispositions to work effectively with people who define problems differently than they do.

ABET EC 2000 Criteria 3 – Student Outcomes approved in 1996; revised 2004 (c) an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability (f) an understanding of professional and ethical responsibility (h) the broad education necessary to understand the impact of engineering solutions in a global and societal context (j) a knowledge of contemporary issues

Methodologies for Producing Globally Competent Engineers in the U.S. University course work (language, culture, histories of cross-cultural or non-U.S. technology development, etc.) Study Abroad Research, Work or Service Experiences Abroad Working on International Teams When U.S. participants travel to another country When U.S.-based participants do not travel to another country Face-to-face Electronically-mediated Working on U.S.-based Teams with an Appropriate Technology (International) focus Other

Example: Purdue Global Engineering Program VISION: “... Leadership through global impact...” The Purdue Global Engineering Program will be an integrated, model program for engineering schools across the world, offering the tools and opportunities for global engineering discovery, education of engineers who exhibit scientific initiative and cultural sensitivity as they step into roles of international leadership, engaging and positively impacting communities both at home and abroad. MISSION: “... Global Leadership in Engagement, Learning, Discovery...” The GEP mission includes the three Purdue mission areas as follows: Engagement: Improve the competence and livelihood of the engineering, academic and business communities in Indiana, the US and the world, through building global partnerships. Discovery: Strengthen the College of Engineering's signature as a global engineering hub for strategic research in targeted geographic areas to meet the global engineering grand challenges of the 21st century and beyond. Learning: Provide opportunities to Engineering students of Purdue through global educational initiatives that empower them to become leaders of engineering discovery, engagement and learning.

Example: Purdue Global Engineering Program The Global Engineering Learning Portfolio is made up of opportunities in the following areas: Study Abroad Global Service Learning Global Design Teams International EPICS (Engineering Projects in Community Service) Engineers Without Borders / Engineers for a Sustainable World Work Abroad Global Research Global Awareness at Home Global Scholarships

Example: Engineers Without Borders

Cal Poly Examples Projects Individual or Group Course Projects Individual or Group Senior Projects Student Groups / Programs Engineers Without Borders International Computer Engineering Experience (ICEX) Program Structural Engineers for Humanity (formerly Structural Engineers for Haiti) Student & Faculty Exchanges Coursework Mandatory US Cultural Pluralism course Diversity Learning Objectives (GE and non-GE) Voluntary Appropriate Technology for the World’s People: Development | Design Global Engineering: Gender, Race, Class, Nation Language

Ongoing Challenges in Producing Globally Competent Engineers

Definition Assessment Program Educational Objectives Student Learning Outcomes Resources To support program development and participation To conduct assessment that is efficient, effective, sufficient, and consistent Scale Funding Voluntary vs. involuntary programs Faculty expertise, prioritization, comfort University structures to support (or not support) implementation mechanisms such as team-teaching, international travel, etc.

Our Project Today Assessment Criteria for Global Competency for Cal Poly Program Educational Objectives (institutional-level) Student Learning Outcomes

Assessing Program Educational Objectives

Reviewed 48 programs in 36 institutions (235 PEOs total) Referred to “global,” “international,” or “world” less than 7% of the time Examples globalized professional environment contribute to the state, nation, or global community respond to global changes function in a global environment remain globally competitive global involvement and awareness prepared for global issues engagement locally and globally aware of impact nationally and globally success in international activities Referred to “culture” or “cultural” 0% of the time

Assessing Student Learning Outcomes This is (even more) complicated …

Assessing Student Learning Outcomes Jesiek, Brent K., Yi Shen, and Yating Haller. (2012). Cross- Cultural Competence: A Comparative Assessment of Engineering Students. International Journal of Engineering Education, 28(1): “cross-cultural competence is a key facet of global competency for engineers” Propose using the Miville-Guzman Universality-Diversity Scale- Short form (MGUDS-S) 15-item instrument that measures Universality-Diversity Orientation: “an attitude of awareness and acceptance of both similarities and differences that exist among people” Original Sources: Miville, M. L., Gelso, C. J., Pannu, R., Liu, W., Touradji, P., Holloway, P., et al. (1999). Appreciating similarities and valuing differences: The Miville-Guzman Universality-Diversity Scale. Journal of Counseling Psychology, 46: 291–307. Fuertes, J. N., Miville, M. L., Mohr, J. J., Sedlacek, W. E., & Gretchen, D. (2000). Factor structure and short form of the Miville-Guzman Universality-Diversity Scale. Measurement and Evaluation in Counseling and Development, 33: 157–169.

Miville-Guzman Universality Diversity Scale-Short form (MGUDS-S) 15-item instrument that measures Universality-Diversity Orientation (UDO): “an attitude of awareness and acceptance of both similarities and differences that exist among people” UDO Axes Diversity of Contact (behavioral) Students' interest in participating in diverse social and cultural activities Relativistic appreciation of oneself and others (cognitive) The extent to which students value the impact of diversity on self- understanding and personal growth Degree of emotional comfort with differences (affective) Students’ degree of comfort with diverse individuals (reverse scored)

Miville-Guzman Universality Diversity Scale-Short form (MGUDS-S) Diversity of Contact 1. I would like to join an organization that emphasizes getting to know people from different countries 2. I would like to go to dances that feature music from other countries 3. I often listen to music of other countries 4. I am interested in learning about the many cultures that have existed in this world 5. I attend events where I might get to know people from different racial backgrounds Relativistic Appreciation 6. Persons with disabilities can teach me things I could not learn elsewhere 7. I can best understand someone after I get to know how he/she is both similar to and different from me 8. Knowing how a person differs from me greatly enhances our friendship 9. In getting to know someone, I like knowing both how he/she is different from me and is similar to me 10. Knowing about the different experiences of other people helps me understand my own problems better Comfort with Differences (reverse-scored) 10. Getting to know someone of another race is generally an uncomfortable experience for me 11. I am only at ease with people of my own race 12. It's really hard for me to feel close to a person from another race 13. It is very important that a friend agrees with me on most issues 14. I often feel irritated by persons of a different race

Assessing Student Learning Outcomes Jesiek, Brent K., Yi Shen, and Yating Haller. (2012). Cross-Cultural Competence: A Comparative Assessment of Engineering Students. International Journal of Engineering Education, 28(1): Comparison of students who opt into global engineering programs (primarily at Purdue University) as compared to the cross-cultural competence of first-year engineering students also at Purdue University “MGUDS-S is an appropriate instrument for assessing the cross- cultural competence of engineering students, especially in the context of global engineering programs” “MGUDS-S also has the advantage of being relatively easy and quick to administer,” along with being “freely available for use its unmodified form”

Assessing Student Learning Outcomes Is the MGUDS-S a tool we wish to recommend/adopt at Cal Poly for assessing global competency? Is the MGUDS-S a tool we wish to recommend/adopt at Cal Poly for assessing the the differential impacts of individual global competency mechanisms/programs/experiences as well as cumulative experiences? That is, can we distinguish, for example, between the impact of a course-based experience vs. participation in Engineers Without Borders or another international program? Or between the impact of a 1-week module vs. a 10-week course? Other questions What are minimum/acceptable global competency goals/standards for Cal Poly graduates? What mechanisms are necessary/sufficient to achieve these minimum/acceptable global competency goals/standards?

Pilot Implementation of MGUDS-S Populations of Students 36 students voluntarily active with Engineers Without Borders 116 senior project students in courses that included some global learning opportunities (not active with EWB) A small number of students participated on international teams and/or completed senior projects with an international focus All students: 4-6 hours of formal cross-cultural training in 2 nd and/or 3 rd quarter 32 students in a required AERO course (not active with EWB)

Results: Universality-Diversity Orientation EWB (n=36)Senior Project (n=116) AERO (n=32) Diversity of Contact Relativistic Appreciation Comfort with Differences Overall UDO =lowest; 6= highest Strongly Disagree (1)Disagree (2)Disagree a Little Bit (3) Agree a Little Bit (4)Agree (5)Strongly Agree (6)

Implications Students who voluntarily participated in EWB at Cal Poly scored higher on each axis of the UDO Supports Jesiek, Shen and Haller’s conclusion that the MGUDS-S is an efficient, effective, sufficient, and consistent instrument for assessing global competency

Next Steps at Cal Poly More systemic integration of MGUDS-S in global competency assessment efforts Identification of minimum/acceptable scores on this scale Longitudinal and statistical analysis of the individual and cumulative impact of globally-focused student experiences on MGUDS-S score More detailed identification and comparative analysis of global competency opportunities/initiatives at Cal Poly Identification of additional assessment mechanisms that focus on skills/application and knowledge, in addition to attitudes

Next Steps at Cal Poly More systemic integration of MGUDS-S in global competency assessment efforts Longitudinal and statistical analysis of the individual and cumulative impact of globally-focused student experiences on MGUDS-S score, including: Implementation of pre-, post-assessments Extension of analysis beyond EWB vs. Non-EWB Identification of the impact of any hidden or secret curriculum that deprioritizes global competence More detailed identification and comparative analysis of global competency opportunities/initiatives at Cal Poly Identification of additional assessment mechanisms that focus on skills/application and knowledge, in addition to attitudes

Next Steps at Cal Poly More systemic integration of MGUDS-S in global competency assessment efforts Longitudinal and statistical analysis of the individual and cumulative impact of globally-focused student experiences on MGUDS-S score More detailed identification and comparative analysis of global competency opportunities/initiatives at Cal Poly Rates of and reasons for student/faculty participation (& barriers) Exploration of the ways in which U.S.-focused programs focused on diversity, inclusivity, and/or equity may prepare/not prepare globally competent engineers Identification of additional assessment mechanisms that focus on skills/application and knowledge, in addition to attitudes

Next Steps at Cal Poly More systemic integration of MGUDS-S in global competency assessment efforts Longitudinal and statistical analysis of the individual and cumulative impact of globally-focused student experiences on MGUDS-S score More detailed identification and comparative analysis of global competency opportunities/initiatives at Cal Poly Identification of additional assessment mechanisms that focus on skills/application and knowledge, in addition to attitudes

Extra Slides

1.10—Defining Culture: List of characteristics of culture (Peace Corps, Culture Matters, p. 233/241) 1. culture is collective, shared by a group 2. culture is learned 3. it has to do with values, beliefs, assumptions, attitudes, and feelings 4. it involves customs and traditions 5. it influences or guides behavior 6. it is transmitted from generation to generation 7. it is unconscious or implicit 8. it is a response/adaptation to reality

Theorizing Culture “ The central focus here is on the relationship between culture in the abstract—the underlying values and assumptions of a society—and culture in the flesh—the specific behaviors that derive from those values. It is important to understand that what people do and say in a particular culture, whether it be yours or that of your host country, are not arbitrary and spontaneous, but are consistent with what people in that culture value and believe in. By knowing people’s values and beliefs, you can come to expect and predict their behavior.” Culture Matters: The Peace Corps Cross-Cultural Workbook, p. 6/14

Culture “ Regarding culture, like most cultural anthropologists I do not restrict the scope of the world to the intangible areas of ideas, religion, arts, literature, language, films, oral traditions, and, in short, the intellectual life of people. Rather, the concept of culture in an anthropological sense has more to do with the total knowledge and way of life of a group of people; both conscious and unconscious, implicit and explicit. It is everything that a group of people has learned, and therefore it is best defined in contrast not to society but instead to nature, to a people’s physical environment … and to their biological inheritance … Thus culture includes not only the intangible beliefs of a people but also the domain of social action: rituals, work, trade, political institutions, family and kinship, and so on. … Culture cannot be reduced to a catalog of values, traits, ideas, behaviors, and institutions. One has to go inside a culture, talk to people, and find out how they see their world and how commonalities cut across different individuals and groups … A cultural perspective implies studying science and technology from the point of view of different groups of people.” David Hess (1995), Science and Technology in a Multicultural World, pp

Engineering, Culture & Nation Culture – Culture Matters, Peace Corps The Concept of Self Personal vs. Societal Obligations The Concept of Time The Locus of Control Nation + Gender + Race + Class + Age + History + etc.