Presentation on theme: "Assessment of Learning in Student Involvement"— Presentation transcript:
1 Assessment of Learning in Student Involvement Diana Sims-HarrisIndiana University-Purdue University Indianapolis
2 Participant Learning Outcomes Following this session, participants will know or be able to:Describe the importance of providing direct evidence of learning in student affairsDemonstrate an example of a departmental mapping of programs and services to learning outcomes, especially where apparently disparate functions are involvedUnderstand dynamics between academic affairs and student affairs at a large urban universityGain insight to the assessment practices of a large department and results of the dataApply information shared to other institutional contexts
3 Session Overview Division of Student Affairs Context Faculty and Student Affairs ContextOSI Assessment Strategies and ExpectationsOSI Learning OutcomesAssessment Measures and ApproachesResults and Use for ImprovementsClosing Recommendations
4 Division of Student Affairs context Development of PULs / PCLs
5 Principles of Co-Curricular Learning Core Communication SkillsCritical ThinkingIntegration and Application of KnowledgeIntellectual Depth, Breadth and AdaptivenessUnderstanding Society and CultureValues and EthicsIntrapersonal DevelopmentInterpersonal Development(case-sensitive)
6 Faculty and Student Affairs context Development of PULs / PCLs
7 Disparate perceptions Different historyDifferent prioritiesDifferent cultureDisparate perceptionsSo what do we havein common?Chad Ahren, Ph.D.Diana Sims-Harris
9 From silos to ladders: Phase 1 Critical elements: values, external pressure, and learningCreating buy-in for the benefits and necessity of collaboration5 min max, then report backKezar, A.J. & Lester, J. (2009). Organizing highereducation for collaboration: A guide forcampus leaders. San Francisco: Jossey-Bass.Chad Ahren, Ph.D.Diana Sims-Harris
10 From silos to ladders: Phase 2 Critical elements: mission, social networks and reward systems/structuresTaking the newfound belief in collaboration and puttingit into action5 min max, then report backChad Ahren, Ph.D.Diana Sims-Harris
11 From silos to ladders: Phase 3 Critical elements: integrating structures, reward systems/ structures, and social networksKeeping collaboration current and valuable requires the redesign of campus structuresChad Ahren, Ph.D.Diana Sims-Harris
12 Student Learning in student life and OLS Course objectivesProgramming goalsStudent Learning Outcomesare the common languageEstablishing the concept of common languageMention here that if you are not dealing in learning outcomes and assessment, now is a good time to start? This is a condition for successful partnering and our relevance is determined by our ability to speak this language and do the true educative work.Chad Ahren, Ph.D.Diana Sims-Harris
13 Course Learning Outcomes Recommended LEAD Program/ExperienceRecognize characteristics of individuals functioning in an effective work group or team.Student Organization Leadership Development (SOLD), Student Organization Retreat, Leadership Foundations, Leadership Consultants, Catalyst, Freedom Rides, Project LeadershipIdentify theories, theorists, and issues associated with organizational behavior.Student Organization Leadership Development (SOLD), Leadership Consultants, Catalyst, Project LeadershipApply understanding of principles and theories relating to small group behavior through projects and assignments.Student Organization Leadership Development (SOLD), Leadership Consultants, Catalyst, Project Leadership, Freedom Rides
14 OSI Strategic Priorities Office of Student InvolvementAssessment of Learning:Strategies and ExpectationsOSI Strategic Priorities
15 OSI Strategic Priorities Social JusticeCivic EngagementLeadershipEqual participationInclusivityDynamics of power and oppressionSocial ChangeCollaborationCommon PurposeSelf-knowledgePositive changeEducationally meaningful serviceCommunity improvementOSI Strategic Priorities
16 Leadership Learning Outcomes 1. Gain personal competencies• Obtain and strengthen leadership skills such as conflict management, communication and dialogue, teamwork, time management, proactive event planning, goal setting, and risk taking• Manage their organizational functions through the use of programming, technology, and physical space• Value co-curricular learning as transferable skills that will complement to the classroom experience2. Develop a sense of personal leadership identity• Define their sense of purpose through the personal exploration of strengths, passions, goals, and abilities• Explore personal cultural identity as it relates to leadership3. Understand how their personal leadership identity relates to working with others• Provide intentional experiences for students to interact with others who are different from themselves• Apply individual experience to develop an even fuller understanding of themselves through their interactions with others4. Build community with the IUPUI and Indianapolis community• Create partnerships with students, faculty, staff and organizations• Develop relationships through social and professional networking opportunities• Participate in idea-sharing as it relates to their organizations, programming, and leadership experiences5. Connect to the IUPUI campus• Have an increased awareness of resources that are provided for students and organizations through The Office of Student Involvement and IUPUI• Use their voice effectively to create and engage in a positive collegiate experience for themselves and other students• Be able to navigate the IUPUI community and university system
17 Social Justice Education Learning Outcomes Interact productively with others who are different from themselvesApply individual experience to develop an even fuller understanding of themselves through their interactions with othersExplore social justice advocacy as it relates to leadershipAchieve awareness of their social identitiesGain knowledge of groups whose identities they do not shareNavigate difference by learning, listening, asking and watchingOutline individual responsibility for action to facilitate changeMulticultural Change Process• Awareness • Knowledge • Skills • Action
18 Civic Engagement Learning Outcomes Civic-Minded GraduateAbility to understand interest, responsibility and personal commitment to service and social issuesUnderstanding how social issues are addressed in societyActive participant in society to address social issuesCollaboration with others across difference (includes diversity, interconnectedness, mutuality, and respect)Benefit of education to address social issues
19 OSI Assessment Structure À la Carte approach to assessment using Strategic Priorities:OSI Assessment Structure
20 OSI Assessment Structure Social JusticeSocial IdentityKnowledge of othersCivic Engage-mentHow social issues are addressedLeader-shipPartnership creationOrganizational managementExample:OSI Assessment Structure
21 OSI Assessment Expectations At least two rigorous assessment projects per functional areaOSI Assessment Expectations
22 OSI Assessment Expectations Learning in addition to or instead of evaluation and satisfactionOSI Assessment Expectations
23 OSI Assessment Expectations Choose projects that are manageable and represent likely opportunities for critical learning and improvementOSI Assessment Expectations
24 OSI Assessment Expectations Keep your assessment practice manageable – integrate into current processesOSI Assessment Expectations
25 OSI Assessment Expectations I promise to make reporting requirements manageable, timely and transparentOSI Assessment Expectations
26 OSI Assessment Discussion Functional areas:Review strategic priorities and outcomesHow will you apply these?OSI Assessment Discussion
27 OSI Assessment Discussion Functional areas:Review strategic priorities and outcomesHow will you apply these?OSI Assessment Discussion
29 Practical Information on Reporting Instruments and design were discussed 1-1 meetings with staff throughout the yearPrograms which covered multiple strategic priorities were encouragedGrowing pains in reporting (expectations not always clear to staff, too much data, timing of analysis)Not all information shared in division report each yearMuch of the data is baselineReporting:First round of analysis completed by Assistant Directors (depending on level of experience)Second round completed by Associate Director and Director of OSIThird round completed by Director of Assessment and Planning
30 Alternative Breaks Principle of Undergraduate Learning Measure (5-point scale of “strongly disagree” to“strongly agree”)ResultsCore Communication and Quantitative Skills (PUL 1)As a result of participating in Alternative Breaks, I am better able to communicate as a team100% indicated “agree” or “strongly agree” (n=23)Integration and Application of Knowledge (PUL 3)As a result of participating in Alternative Breaks, I understand the root causes of the social issue that my trip worked withAs a result of participating in Alternative Breaks, I realize that I cannot effectively make social change without being educated on a social issue96% indicated “agree” or “strongly agree” (n = 22)65% indicated “agree” or “strongly agree” (n = 15)Values and Ethics (PUL 6)As a result of participating in Alternative Breaks, I have the desire to make a difference in my community
31 Alternative BreaksParticipant comments that support Values and Ethics:I gained insight on what a small committed group of motivated people can do for a reason.The strengths that I gained from this trip are determination and responsibility to make more of a difference with the social issues going on right in my community. The knowledge of knowing that I have the power to make a change to help others in my community to have some hope for the future.I learned to be more open minded about the social topics involved.I feel as though I have realized more of how compassionate I strive to be. I realize that I want to continue my services in the community, and I truly want to work in a non-profit educational facility now also. This trip made me realize a lot about what populations I want to be working with in the future.
32 Freedom Rides Lead IUPUI Learning Outcome Measure (4-point scale of “strongly disagree” to “strongly agree”)ResultsGain Personal Competencies (LO1)Freedom Rides helped me develop individual goals.95% indicated “agree” or “strongly agree” (n=21)Develop a Sense of Personal Leadership Identity (LO2)Freedom Rides helped me recognize how groups and communities I belong to affect my leadership style.Freedom Rides helped me create a vision statement.Freedom Rides helped me identify personal leadership skills and strengths89% indicated “agree” or “strongly agree” (n=63)Understand how their personal leadership identity relates to working with others (LO3)Freedom Rides helped me identify social justice issues of personal importance.Freedom Rides helped me compare different social justice leaders' approaches to leadership.100% indicated “agree” or “strongly agree” (n=42)
33 Freedom RidesParticipant comments that support the Personal Leadership Identity in Relation to Others outcome:Has the trip helped you learn about social justice? If so, how?Yes, it has helped me to know a change must happen all people deserve to be treated fairly.Yes, I learned that everyone deserves social justice no matter what their background may beIt taught me that it affects one and that we all have the power so don't be afraid to speakYes! I already knew a fair amount about social justice from some courses, but I learned even more!By walking through the journey taken by the leaders
34 AFLV – Program Evaluation Principle of UndergraduateLearningMeasure(strongly agree, agree, neutral, disagree, strongly disagree)ResultsCritical Thinking· AFLV provided me with skills to generate new ideas or ways to improve things in my chapter and at IUPUI.96% indicated “agree” or“strongly agree” n=25)Values and Ethics· AFLV helped me recognize my personal values and ethics.100% indicated “agree” orIntegration and Application ofKnowledge· AFLV provided me with skills to develop individual/organizational goals.
35 AFLV – Focus GroupSelected participant comments that support Integration and Application of KnowledgeEverybody's chapter is different and you can come back and take some of their ideas, like their bigger successes and take those back, and get the best aspects of all different kinds of Greek life in all different kinds of communities and bring it back and apply it hereI wish we could all break down the distance between IFC fraternities and everyone else... Just like at AFLV, they accommodated everybody. That is a good step forward in doing that. Just being IUPUI students and being in Greek organizations, we should take the reins from that and see how they expanded on it, and do it ourselves, essentiallyWe are only now starting to have to deal with connecting with alumni, so we just started doing like a letter that we are putting together, like a newsletter type thing that talks about the accomplishments of the chapter for the year and once we realized that you know it does not have tojust go to alumni so we are going to send it to parents for like PR and stuff like that and show what they are doing...
36 Academic Organizations Survey Principle of Undergraduate LearningMeasure(4-point scale of “strongly disagree” to “strongly agree”)ResultsCore communication and quantitative skills (PUL 1)Being involved with this organization provided me with the skills to communicate ideas and information92.78% indicated “agree” or “strongly agree”Integration and application of knowledge (PUL 3)Being involved with this organization provided me with the skills to make connections with students, faculty, and staff on campus.93.30% indicated “agree” or “strongly agree”Values and ethics (PUL 6)Being involved with this organization provided me with the skills to recognize how groups and communities I belong to influence my leadership style.87.63% indicated “agree” or “strongly agree”
37 Academic Organizations Survey Being involved in this organization allowed me to use information I learned through my coursework:Always8.25%Quite a bit27.84%Sometimes40.21%Very Little23.71%Being involved with this student organization I have been able to connect my experiences to the Principles of Undergraduate Learning:(n=162)Core Communication and Quantitative Skills18.69%Critical Thinking16.26%Integration and Application of Knowledge15.35%Intellectual Depth, Breadth, and Adaptiveness12.16%Understanding Society and Culture19.60%Values and Ethics17.93%
38 Assessment Loop1. Set goals,ask questions2. Gather evidence3. Interpretfindings4. Use forimprovementRAOverview of our workResearch is different than assessmentLeskes, A. & Wright, B. (2005). The Art and Science of Assessing General Education Outcomes: A Practical Guide. Washington, DC: AAC&U Publications.
39 Use for Improvement – OSI wide Results indicate need for more opportunities for deep reflection so students can connect experiences with outcomes.There is a need for more collaborations across functional areas. Shared language from the strategic priorities can help facilitate this.Some direct measures of assessment should be used; these measures would be a nice comparison to the current self-reported data.Need for deeper reflection - As an example, students in academic organizations demonstrate understanding of the connection when asked about specific aspects but when asked about a general link between the club and the PULs, less than a fifth of them agreed that there was a connection.
40 Use for Improvement – Lead IUPUI Results indicate consistency in reported learning with different groups of students overtime.As an action item from previous assessment reports, more long-term experiences have been developed and show positive results. A residential based learning community is being explored.The Leadership Foundation program saw positive learning outcomes but spotty attendance and at times, an inconsistent message. The series was restructured as a cohort program with more direct staff facilitation.
41 Use for Improvement– Lead IUPUI The additional qualitative measures allowed for a deeper understanding of student learning. More guided reflection should be explored.Many students participate in several Lead IUPUI programs, but assessment does not capture the possible cumulative effect of participation in multiple programs.Quantitative feedback for SOLD shows a slight decline, but the qualitative results showed some of the strongest connection to learning yet. Strategies are being used to look for data trends and examine possible explanations for differences.(daily reflections for experiential trips and focus groups for extended programs) from previous reports
42 Closing Recommendations Know your institution culture and context; what is your collective mission?Find allies with common interests and prioritiesTell your story, use shared language in learning outcomesStart small and grow; integrate into current processesMake obvious, easy improvementsHelp students make connections and document them
43 Discussion/Questions/Contact Diana Sims-Harris, M.S.Ed. Associate Director & Chad Ahren, Ph.D. Director Office of Student Involvement Indiana University-Purdue University Indianapolis (317)Robert W. Aaron, Ph.D. Director, Assessment and Planning Division of Student Affairs Indiana University-Purdue University Indianapolis (317)