1. Fostering Global Citizenship in Higher Education Assessment and Evaluation

2. Session Agenda Topics Setting the context Defining competencies Assessing growth Contexts Graduate Undergraduate

3. Objectives identify ideas on how to implement assessment & evaluation techniques at the program and degree levels review how developing competencies are linked to program design and measurement understand the merits and challenges of different methods and timing of data collection; and implement how to use what has been found in future program design

4. …a number of inter-related trends at the macro and mezzo level currently are affecting program design throughout higher education in the US.. Among those trends are:  An increase in short-term, faculty-led programs to accommodate new majority students, professional school students, and other student groups new to education abroad  An increase in required pre-departure, re-entry, and embedded programs  An increase in home campus or sponsoring agency intervention in the experience to support/direct students’ learning  An increase in student/parent expectations for student support services that mirror those found in the US, whether or not such services are typically provided to students in the host country  An increase in education abroad being part of a campus-wide plan for internationalization  An increasing emphasis on program evaluation, particularly outcomes assessment  An increase in the emphasis on economic rationales for education abroad, consumerism, and education abroad in its job preparation context Rodman & Merrill, 2008 Developments in “the academy”

5. Classroom-basedTour-based Experience-based embedded grand tour hybrid direct enroll service learning service branch campus study center workinternships short-term group excursion independent study language training volunteerism adventure field-based mission work Situating the Changing Study Abroad Program Models

6. Classroom-basedTour-based Experience-based embedded grand tour hybrid direct enroll service learning service branch campus study center workinternships short-term group excursion independent study language training volunteerism adventure field-based 1950s 2000s 1960s 1970s SIETAR Intercultural Communications Literature Maturation mission work Study Abroad Program Model Development & ICC Maturation

7. SIT/PIM Definition of Competency Describe outcomes of the learning process. –action-oriented statements –tell students what they will be able to do, to know, or be aware of by graduation. – related to their professional and personal lives.

8. Competencies are found in the following domains: Knowledge (Cognitive) –Theories, concepts, and models the student can discuss Skills (Behavioral) –What the student can do Self-awareness (Affective) –The awareness of personal values, attitudes, and attributes, and how they connect the individual to their larger environments.

9. Competency delivery flow through phases of the program: On campus format: –Prior experience + on-campus program core courses and educational activities + degree courses + reflective practice phase+ capstone + language and cultural requirement Low Residency format: –Prior experience + summer residency + online coursework with reflective practice + summer residency + online coursework with reflective practice + capstone + language and cultural requirement

10. Program Competency Areas 1.Theory and Practice of Learning 2.Professional Development 3.Identity, Culture, and Communication 4.Reflection and Inquiry 5.Leadership 6.Social Change Processes 7.Teams and Organizations 8.Community and Global Awareness 9.Degree specific competencies (e.g., for International Education program design)

11. Community and Global Awareness Global and community awareness is based on our respect for and appreciation of the interconnections of individuals and systems in our environment and on the planet. This awareness includes one’s capacity to: Identify the issues and their impact on one’s own and others’ communities Make informed choices regarding personal community involvement, social justice issues and leadership roles Recognize the value of multiple perspectives in community and global life Understand macro level political and social theories on community and global awareness Identify the role of conflict transformation, power, and ethics Articulate the way in which technology affects interconnections of local and global communities

12. Name of competency Current status of delivery Review core course objectives, campus workshops & activities, etc. Value to mixed degree student group(1,2,3) *below When deliveredOther potential places to deliver Measurement 1-LearningTheory and Practice Learning styles assessment,  all program phases as articulated through students’ learning plans Early in Fall term Orientation workshop 2- Professional Development Workshops career development,  On-going classroom presentations, course and capstone research,  Professional contacts with faculty, alumni, staff,  Courses-OB1, practitioner inquiry, Training Design  RPP 8-Community & Global Awareness Courses- ICC, Social Change, On-campus extra-curricular groups RPP 2Fall 2 course, Spring 1 course Year long practicum Orientation workshop, speakers across degrees during the academic year 1=necessary 2= highly desirable 3-nice but not required

13. Evaluation is “any effort to use assessment evidence to improve individual, departmental, divisional or institutional effectiveness.” Upcraft, 2003 What do we mean by “evaluation”?

14. Assessment is “any effort to gather, analyze and interpret evidence, which describes institutional, divisional, or agency effectiveness.” Upcraft, 2003 What do we mean by “assessment”?

15. Assessment and Evaluation are tied in with Design From Trochim

16. Gardner’s 5 evaluation frameworks (1994) Professional judgment Measurement Decision-oriented evaluation Goal-free/responsive evaluation Assessment of congruence between performance and objectives

17. A Formative and Summative Process of: Identifying goals, objectives or standards of performance Identifying or developing tools to measure performance; Comparing measurement data collected with previously identified objectives or standards. (Gardner, 1994)

18. Assessment of congruence between performance and objectives IndividualProgram What competencies (knowledge, attitudes, skills, awareness) will the individual learner develop? What does the program (institution) hope to accomplish by this program? Examples? From whose perspective?

19. Methods Choice depends on: Scope of assessment Values or paradigm (you, the institution)

20. Scope of the Assessment Institution Program Course Student

21. Institutional Context Who are the decision makers inside and outside of your program or institution? What do they value?

22. Challenges of assessment and evaluation Conflicting needs and values Range of methodological approaches Criteria ambiguity or disagreement Difficulty in attributing change Overzealous desire to measure Complexity Controversy Clarity

23. Best Practices of Assessment and Evaluation Involve all stakeholders at all stages. Determine individual meaning of assessment elements Clarify purpose, timing, uses and appropriate approaches Identify possible “intervening factors” Triangulate methods and participants. Work ethically Sort out “measurability”

24. Group Discussion Application Questions